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Foreword 


I am pleased to table this September volume of my 1996 Report comprising chapters 13 through 20, 
issued separately. Bound with this Foreword are the Main Points of the chapters, which include: 


es Accountability Practices from the Perspective of First Nations 

14. Service Quality 

ley Federal Science and Technology Activities: Follow—up 

16. Treasury Board Secretariat - Renewing Government Services Using Information Technology 

LF, Human Resources Development Canada — Canada Pension Plan: Disability 

18. Revenue Canada and Department of Finance — Excise Duties and Taxes on Selected Commodities 
19: Revenue Canada — Child Tax Benefit and Goods and Services Tax Credit Programs 

20. Revenue Canada — Creating One Revenue Canada: The Administrative Consolidation of 


Customs and Excise and Taxation 


Emerging from the recent government—wide Program Review have been several initiatives aimed at 
improving services to the public while streamlining their delivery, as departments and agencies attempt to manage 
the effects of downsizing and budget reductions. They have been given more authority and flexibility to run their 
operations and, at the same time, more responsibility to account for results. The Treasury Board Secretariat’s 
approach has been moving from a focus on controls and compliance to an emphasis on providing co-ordination, 
encouragement and support to departments and agencies. Our recommendations, whether addressed to 
departments or to the Secretariat, have taken into account this changing relationship. 


As our chapters note, several of the initiatives we examined have produced measurable improvements in 
service and others promise to do so as they mature. Certain departments have also had some success in 
streamlining their services, thereby reducing overlap and duplication. However, progress so far is not consistent 
across the government, nor have all departments managed with equal effectiveness the changes they are 
undergoing. We believe that our recommendations to specific organizations will be useful to others who are still 
in the early stages of implementing some of the initiatives. 


In the climate that prevails across the government today — with its emphasis on innovation, initiative, 
flexibility and service to program clients against a backdrop of downsizing, fewer controls and a focus on 
obtaining measurable results — it is particularly important that certain fundamental controls continue to be 
exercised. As some of our chapters note, the failure to do so exposes the public purse to considerable risk. 
Moreover, we note that some departments cannot assess the magnitude of that risk because they do not 
systematically gather the kind of information needed to do so. Nor have they determined whether the risk has 
become a reality. Implementing our recommendations would be a significant step in closing this information gap 
so that senior managers in departments can demonstrate and be accountable for the results they obtain with public 
money. 
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Study of Accountability Practices 
from the Perspective of 
First Nations 


Assistant Auditor General: Maria Barrados 
Responsible Auditor: Ronnie Campbell 


Chapter 13 — Main Points 


13.1 As the relationship between the federal government and First Nations evolves, the issue of accountability 
continues to present difficulties to all parties. In its accountability to Parliament, government is expected to report 
on activities undertaken and results achieved. This relationship often becomes more complicated where third 
parties are used to carry out activities for which government remains responsible. 


13.2 This study focusses on the perspectives of one set of the parties in such a relationship, the First Nations. It 
is aimed at improving our understanding and encouraging the pursuit of solutions. The Office intends to build on 
this understanding and to take into consideration the lessons learned for future audits and studies dealing with the 
relationship between First Nations and government. 


13.3 This study is based on interviews with nine First Nations and one Tribal Council that Indian and Northern 
Affairs Canada considered to be well managed. Although there is a broad legal framework that governs First 
Nations’ dealings with the federal government, and First Nations have obligations to their own communities, the 
discussion focussed on First Nations’ day-to-day interaction with government departments. Management practices 
of First Nations and of government that participants felt could be improved were discussed at length. 


13.4 Participants recognized the importance of effective accountability and articulated a clear sense of 
essential accountability factors. For example, participants felt that it is essential that both First Nations and 
government have clear and commonly held objectives, that audit meet the needs of their communities as well as 
of government, and that the focus be on results as opposed to process. Taken together, these factors fit within most 
definitions of accountability. In an area as complex and contentious as this, it is encouraging to see that these First 
Nations hold views that appear, to some degree, to be consistent with such definitions. 


i Pads) However, participants strongly emphasized a two-way perspective as an essential ingredient of 
accountability. Much of what we heard spoke of accountability among partners or equals. Generally, 
accountability in government is viewed as a hierarchical relationship involving a delegator, who assigns 
responsibility to another — a delegate. It is evident that there is a significant difference between this concept and 
the one that participants felt would better suit their needs. 


13.6 While participants felt the current situation was unsatisfactory, many were optimistic and felt that 
progress could be made. They recognized that both they and government officials work under difficult and 
uncertain circumstances. This study represents one step toward encouraging improved dialogue with government 
and First Nations as they develop practical approaches to strengthening and clarifying accountability relationships. 
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Service Quality 


Assistant Auditor General: Maria Barrados 
Responsible Auditor: Theresa Duk 


Chapter 14 - Main Points 


14.1 The federal government delivers many services directly to the public, ranging from issuing passports to 
answering tax enquiries, to processing claims for employment insurance. However, recent studies continue to 
show that Canadians remain generally dissatisfied with federal services. 


14.2 Since 1990, the government has committed itself to establishing service standards and being accountable 
for the resulting performance. Service standards would clearly specify the quality of service Canadians can expect 
to receive, including its timeliness, accessibility and accuracy. An original goal of the government’s was to have 
standards published for major services by March 1994 and to report performance in 1994—95. Later, the goal was 
extended to the end of 1995. 


14.3 Our audit of 13 highly visible services revealed that the government’s expectations have not been 
realized. As of 31 March 1996, although many of the 13 services had put in place some elements of the concept, 
none of them had implemented service standards that met all the requirements. 


14.4 Information on clients’ priorities is a prerequisite for developing sound service standards and 
improvement plans. Only about half the services we audited have collected information on client priorities and 
their relative importance. Even fewer have assessed their clients’ satisfaction with the way each of their priorities 
is served. This knowledge is important; there is little point to investing effort in areas where client priority is low 
and satisfaction high when there may be a much better return in areas of high priority and low satisfaction. We 
also found that many services have not determined the full costs of delivering service. Cost information helps to 
ensure that government clients receive high-quality and affordable services. 


14.5 Telephone services are a significant and growing method of contact between Canadians and the 
government — with more than 30 million calls answered per year in the six operations we examined. However, 
the quality of government telephone services needs attention. In one department we found that during peak 
periods as many as 19 out of 20 calls receive a busy signal. Few departments check the accuracy rate of 
information that agents give to callers — one that has done so found that the accuracy rate falls between 60 and 80 


percent. We believe that departments could and should apply service standards to remedy these persistent 
problems. 


14.6 Much in the area of service standards remains to be done, and a sustained effort is required. Deputy 
ministers need to provide leadership to ensure that departments focus sufficient effort on major services and 
follow appropriate plans in developing and implementing service standards. So far, little information has been 
provided to Parliament to indicate clearly the progress against implementation goals. The government needs to 
improve its reporting to Parliament on this important subject. 
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Federal Science and 
Technology Activities: Follow-up 


Assistant Auditor General: Robert L. Lalonde 
Responsible Auditors: Richard Flageole and Jacques Goyer 


Chapter 15 — Main Points 


15.1 The federal government released Science and Technology for the New Century —A Federal Strategy in 
March 1996. The Strategy was accompanied by departmental action plans and a Framework for the Human 
Resources Management of the Federal Science and Technology Community. These initiatives, aimed at addressing 
certain government-wide concerns noted in our 1994 Report, represent important steps in the right direction. 


1532 We found that science-based departments and agencies are making good progress in focussing their 
science and technology activities and setting their priorities. Although priorities at the government-wide level 
have not been enunciated, a mechanism has been established to make recommendations on them to Cabinet. 
Furthermore, the Strategy provides direction to departments and agencies in the form of operating principles and 
directions for conducting many aspects of their science and technology activities. The challenge will be to ensure 
that such principles and directions become an integral part of the day-to-day management of activities in 
science-based organizations. 


15.3 The Strategy gives full recognition to the importance of departments and agencies working together and 
increasing intergovernmental co-operation and co-ordination. Various initiatives are also under way to improve the 
assessment of the results and impacts of federal science and technology activities, and to report on the growth of 
Canada’s knowledge-based economy. 


15.4 The positive features of the Framework for managing scientific personnel include consultation with and 
participation of a significant number of stakeholders, leadership of work groups by line-department officials to 
maximize commitment and ownership, and clearly stated expectations of results. People interviewed believe that 
work under way addresses issues of critical importance, and that if recommendations are successfully 
implemented, the outcome could be the resolution of many of the issues we reported in 1994, such as the 
systematic renewal of scientific personnel and the maintenance of the skills and knowledge base of research 
establishments. Other initiatives conducted in parallel to the Framework, such as modifications to the Work Force 
Adjustment Directive, have also had positive effects. 


bee) Although progress is being made, we have some concerns, notably about the absence of a forum to share 
“best practices” and the little attention given to the issue of performance measures and reporting mechanisms for 
the management of scientific personnel. 


15.6 Although we are encouraged by the progress reflected to date in the Strategy and the Framework, 
implementation is the real challenge. For this reason, we believe that the government needs to devote considerable 
attention to establishing results-oriented, time-phased implementation plans for both the Strategy and the 
Framework. The government also needs to ensure that accountability for results is clearly established. 
Implementation will require leadership and perseverance at all levels of government — from ministers to 
scientists. Parliamentary oversight also needs to continue. 
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Treasury Board Secretariat 


Renewing Government Services 
Using Information Technology 


Assistant Auditor General: David H. Roth 
Responsible Auditor: Nancy Cheng 


Chapter 16 — Main Points 


16.1 Government expenditures on information technology, including capital, operating, personnel and other 
related costs, represent a major investment of over $3 billion annually. Using information technology judiciously 
is recognized as a prerequisite for renewing government services. 


16.2 We examined two sets of initiatives relating to the government’s Blueprint for Renewing Government 
Services Using Information Technology. We noted that implementation of the Blueprint framework is starting to 
gain momentum and that visible progress is being made in moving toward shared administrative systems. At the 
same time, we also identified several areas for improvement. For example, we noted that the Treasury Board 
Secretariat focusses on reporting the status of flagship projects but not the results they have achieved. Reporting 
their results could promote departmental participation and build on the Blueprint’s existing momentum. There is a 
need for more timely development and implementation of government-wide network management to support the 
sharing and interoperability of systems. We observed some lessons that can be learned in the use of technology 
funds to encourage partnership and innovative use of information technology. 


16.3 There has been significant progress since the establishment of the Chief Informatics Officer function at 
the Secretariat in 1993. The release of the Blueprint framework in 1994 was a key milestone in the move toward a 
government-wide perspective of technology. Furthermore, the use of an interdepartmental committee structure and 
related work groups has encouraged dialogue and fostered co-operation in the information technology and user 
communities within government. 


16.4 Yet many challenges remain. To harness and build on the existing momentum of support for service 
renewal, the Secretariat needs to exercise corporate governance and accountability through strategic oversight, 
without jeopardizing the consensus-building process that fosters horizontal co-ordination, and co-operation across 
departmental lines. 
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Human Resources 
Development Canada 


Canada Pension Plan: Disability 


Assistant Auditor General: David Rattray 
Responsible Auditor: Louis Lalonde 


Chapter 17 — Main Points 


17.1 Our audit examined the adequacy of management practices related to assessing eligibility for Canada 
Pension Plan (CPP) Disability benefits and measurement and reporting of achievement of objectives. Our 
observations and recommendations for improvement are reported in this chapter. 


172 Over the past 10 years, CPP Disability benefit payments have more than tripled, climbing from 

$841 million in 1986-87 to close to $3 billion in 1995-96. The number of beneficiaries has almost doubled over 
the same period, from 155,000 to 300,000. The increase in Disability payments has had an important impact on 
the increase in the most recent projected pay-as-you-go rate. 


17.3 Important improvements need to be made to management practices related to the eligibility of Disability 
benefits. Moreover, CPP management does not have complete and relevant data that would enable it to manage 
eligibility with due regard to economy. We noted that: 


e management information systems are limited and do not allow for sound analysis. Variances between 
actuarial estimates and actual results have not been examined and reported regularly; 


e applications are often adjudicated on the basis of information obtained from the applicants and their 
physicians. No analysis has been carried out to determine at what stage in the process it would be 
most economical to search for additional information; 


e quality control necessary to improve the soundness of disability adjudication is neither formal nor 
systematic; 


e the effect of guidelines on the number and profile of new applications accepted is not determined; 
e few beneficiaries are subject to reassessment for assessing continuing eligibility; and 
e there is limited exchange of information between the CPP and other disability plans. 


17.4 In the last decade, the CPP has faced an unprecedented increase in the number of disability claims, as 
have other public and private disability plans. CPP administration undertook a study to identify the reasons for the 
increase, although no quantification was done. Several measures undertaken to respond to the concerns raised in 
the study were: 


© new incentives to reduce barriers to employment of CPP Disability beneficiaries; 
e revised Disability determination guidelines; 
e improved communication with clients; and 


e improvement of the reassessment program. 
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Chapter 17 — Main Points (cont'd) 


CPP management is starting to come to grips with determining more clearly the causes of growth in Disability 
payments. The number of beneficiaries has declined recently, for the first time in several years. Many of the new 
initiatives show promise for the future. We commend all those involved for their efforts to improve the situation. 
It is too early, however, for the Office to comment on the success of these endeavours, as the number of appeals 
has grown steadily at all levels for the same time period. 


Ny es Guidelines introduced over the years have had a significant impact on costs, which are reflected 
eventually in the long-term projected contribution rate. While legislative changes require estimates of the impact 
of contribution rates, and formal consultation with the provinces, we are concerned that some important changes 
to Disability eligibility practices were introduced in the past without proper assessment of projected costs or 
actuarial impact analysis. 


17.6 The actual cost of Disability benefits exceeded actuarial estimates for the period reviewed until 1995. 
The roles and responsibilities of the CPP and the Chief Actuary of the Office of the Superintendent of Financial 
Institutions for ensuring the reliability and integrity of the data used for actuarial projections are unclear. 
Furthermore, we believe that the Department would benefit from having in-house actuarial expertise to improve 
the exchange of operational and strategic information. 


17. The Canada Pension Plan was created in 1966 as a mandatory social insurance program to provide some 
measure of protection to Canadian workers and their families from loss of earnings resulting from retirement or 
death, as well as from serious and prolonged disability. CPP Disability provides, on average, about half of 
recipients’ incomes; this share is partly due to the fact that some other income providers reduce their benefits by 
the amount of CPP Disability benefits. This makes the Plan a first payer in most situations. 


17.8 There is limited exchange of information between the CPP and provincial plans or private insurers on 
medical assessment, decisions rendered, rehabilitation, follow-up and termination of payment to beneficiaries who 
are receiving benefits from more than one of these organizations. However, the Act that governs the disclosure of 
information considerably limits the exchange of information. Because there is little co-ordination, beneficiaries 
often provide the same information to several plans, which results in unnecessary duplication of effort and cost. 
Better co-ordination among the plans would not only improve service but also reduce expenditures for all plans. 


17.9 Recent evaluation studies provide useful information about beneficiaries and alternative ways of 
providing public disability insurance. Overall, the issues and questions addressed by the studies were appropriate 
and extensive. The evaluation’s Interim Report presented a number of proposals for improving adjudication and 
appeals processes. The fact that CPP Disability does not have precisely defined objectives has hampered the 
evaluation’s measurement of program success. More discussion and clarification of what CPP Disability is 
expected to accomplish would be desirable. In addition, more information is needed to cover the significant 
performance information gaps. The Department has yet to take sufficient steps to fill the identified gaps. 


17.10 Our audit did not attempt to determine whether the CPP is a better plan than the Quebec Pension Plan 
(QPP) from the viewpoint of either the beneficiaries or the contributors. CPP legislation requires that comparable 
benefits be provided by the Canada Pension Plan and the QPP. The legislation allows a province to opt out of the 
CPP provided that the province sets up a comprehensive pension plan that provides benefits comparable with 
those provided by the CPP. According to CPP management, changes to both the CPP and QPP over the years have 
resulted in differences that at times have increased the distinctions and at other times narrowed them. It is 
therefore important to evaluate the differences between these plans and make adjustments where appropriate. 


Important differences between the Disability programs of the two plans have been identified, which may explain 
the increasing disparity between utilization rates. 
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Chapter 17 — Main Points (cont’d) 


17.11. We recognize that determination of disability is a very difficult task. However, every effort should be 
made to minimize subjectivity in the process. In the context of regionalization, closer monitoring and proper 
guidance would assist adjudicators in making fair and consistent decisions. Tighter controls on the quality of 
decisions and greater emphasis on operational controls, such as reassessment, would also reduce the risk of paying 
benefits to persons no longer eligible. It is important that CPP management have appropriate controls in place to 
ensure that those, and only those, who are eligible for benefits are accepted and/or continue to receive benefits and 
that all beneficiaries are treated equitably and in compliance with the legislation. 


17.12 We acknowledge that the issues we have raised in this chapter will be difficult to resolve and cannot all 
be addressed at once. Those involved in administering CPP Disability would benefit from a comprehensive action 
plan that sets out the accountability, priorities, responsibilities and resources, if required. Such a plan would foster 
a common mindset on the issues. The number of recommendations proposed implies that additional resources may 
be required, although the success of many ongoing projects will depend to some extent upon other projects being 
implemented concurrently. We are convinced, however, that significant savings in program costs can be achieved 
as a result of additional efforts to improve the administration of CPP Disability and without causing any prejudice 
to applicants who meet the eligibility criteria of the Plan. 
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Revenue Canada and 
Department of Finance 


| oe Excise Duties and Taxes on 
a sora uit SPIE CEC COMImOG INES 


Chapter 18 - Main Points 


18.1 In 199495, excise duties and taxes generated some $6.9 billion, or six percent of gross federal revenue. 
Of that, 98.4 percent came from the commodities covered in this audit — namely, tobacco, alcohol, motive fuels 
and jewellery. Various estimates suggest that this excise revenue could have been as much as $500 million higher 
were it not for evasion. 


18.2 As part of its 1994 anti-smuggling initiative, the government lowered tobacco taxes and increased 
enforcement resources. Despite the tax reduction and the more vigourous and co-ordinated enforcement efforts of 
Revenue Canada and the RCMP, evasion of commodity taxes persists. Because of differences in the prices of 
cigarettes between provinces that lowered their tobacco taxes in response to the federal strategy to combat tobacco 
smuggling and those that did not, interprovincial smuggling of cigarettes has emerged as a new problem. 


18.3 The 1994 Budget Plan forecast that the anti-smuggling initiative would result in a fiscal cost of 

$300 million in tobacco tax revenue in 1994—95. Federal revenue from tobacco taxes declined by over 

$600 million in 1994-95 compared with 1993-94. Finance’s analysis so far has not determined the extent to which 
this decline is attributable to the reduced tax rate, to evasion or to a change in demand for tobacco products. 


18.4 The inherent nature and underlying causes of evasion need to be analyzed more regularly to provide 
current information to help develop policy and administrative solutions. As well, given the potentially high risk of 
revenue loss from evasion of motive fuel taxes (which yield the largest amount of federal excise revenue), 
Revenue Canada and Finance need to systematically assess the extent of fuel tax evasion. 


18.5 In the past five years, Revenue Canada has done no excise tax audits of most large licensees (who remit 
most of the excise taxes) and so risks failing to collect significant amounts of revenue, as audits are statute-barred 
after four years. The Department recognizes the situation and plans to correct it. 


18.6 Revenue Canada maintains high-level aggregate information such as total revenues and number and 
value of seizures. Finance analyzes the high-level information from Revenue Canada, as well as from Statistics 
Canada, the RCMP and other sources, to monitor excise revenues. However, Revenue Canada does not on a 
regular basis collect detailed information needed to monitor, assess and be accountable for its efficiency and 
effectiveness in administering the commodity taxes and to help Finance’s monitoring activities. 


18.7 Over the past 21 years, Finance and Revenue Canada have conducted studies and reviews of the Excise 
Act. These reviews have demonstrated the need to revise the Act. The departments are currently doing another 
joint review. It urgently needs to be completed, as the delay is holding up necessary changes and other desired 
improvements in the Act and its administration. 
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Revenue Canada 


Child Tax Benefit and Goods and 
Services Tax Credit Programs 


Assistant Auditor General: Shahid Minto 
Responsible Auditor: Basia Ruta 


Chapter 19 — Main Points 


19.1 Revenue Canada administers two key social programs: the Goods and Services Tax Credit and the Child 
Tax Benefit programs. Both are delivered through the tax system. Prior to September 1995, Revenue Canada 
administered child tax benefits jointly with Human Resources Development Canada (HRDC). The latter was 
responsible for determining basic eligibility and Revenue Canada was responsible for all other aspects of the 
program. 


19.2 Each year under these programs about $8 billion in benefits flows tax-free to over 8 million recipients. 
Most are families with low or modest incomes. 


19.3 Revenue Canada faced a significant challenge to launch the GST credit on time in 1990 and the child tax 
benefit in 1993, given the short lead times available to meet statutory deadlines. For child tax benefits, the 
challenge was shared with HRDC. Notwithstanding some important weaknesses in the initial systems, we were 
impressed with the ability of both departments to publicize the programs and meet the statutory deadlines. Since 
the programs were launched, millions of families and individuals have been receiving monthly or quarterly social 
benefit payments. To date, over $30 billion has been paid out under these programs. 


19.4 Revenue Canada has been working to improve service and reduce costs by streamlining its administrative 
processes. Despite some successes, we have four concerns about the administration of these programs. First, we 
found that service needs to be substantially improved in some key aspects of delivering the programs, namely in 
providing access to the Department by phone and in processing child tax benefit applications. Second, the control 
environment needs to be significantly strengthened to provide reasonable assurance that only those who are 
eligible receive benefits and receive them in the correct amounts. (This was also the case with HRDC while it had 
responsibility for determining basic eligibility for child tax benefits, until August 1995.) Third, Revenue Canada 
has not developed a formal strategy for detecting and preventing fraud and abuse in the programs. Fourth, the 
Department does not monitor the programs to assess whether the total amount of benefits flowing from each 
program is reasonable in the aggregate, considering changes in population characteristics and family structure. 
Our analysis of program statistics against independent data on key variables suggests potentially serious problems 
that require immediate attention. 


19.5 The Department of Finance is responsible for assessing the extent to which program policy is producing 
the desired outcomes. Despite their social significance and the dollars involved, neither program’s performance 
has been evaluated against its stated objectives. Further, neither program is scheduled for evaluation in Finance’s 
current three-year plan. 


19.6 Although child tax benefits and GST credits are key components of Revenue Canada’s income 
redistribution line of business, Parliament receives little information on the Department’s performance and 
priorities in administering the two programs. The little information that is reported in the Department’s Part III of 
the Estimates needs to be strengthened and improved. 
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Revenue Canada 


Creating One Revenue Canada: 
The Administrative Consolidation of 

ssistant Auditor General: Shahid Minto : \ 

lca fe8 Jim pte Customs and Excise and Taxation 


Chapter 20 — Main Points 


20.1 Beginning in 1992, the formerly separate departments of Customs and Excise and Taxation were brought 
together into one department, Revenue Canada. Our audit of the “administrative consolidation” of Revenue 
Canada selected three aspects on which to judge success: 


® corporate management of administrative consolidation; 
e the introduction of a single business number for each business client; and 
e the opening of “business window” service sites across the country. 


20.2 Our examination found there were good practices common to the three aspects that helped make their 
implementation successful. These included: 


e aclear overall vision and principles to guide the process; 

e@ the commitment and involvement of senior management; 

e good communication with stakeholders and employees at all levels; and 
® good project management. 


These practices were particularly important given the special difficulties involved in administrative consolidation, 
including an uncertain legislative environment and the challenge of combining different work cultures. 


20.3 A number of areas still require improvement, including levels of service and classification of staff in the 
business window activity. The business number project also faces a major challenge in J anuary 1997, when all 
remaining businesses will be required to convert to the new system. 


20.4 Administrative consolidation has been and continues to be a challenging task and, for the most part, it has 
been well managed by the Department. On the whole, with respect to the aspects we examined, we conclude that 
administrative consolidation is establishing a solid foundation for: 


e streamlined operations; 
e improved service; and 


e enhanced compliance. 


It has also made possible new ways of using and integrating information, both internally and in co-operation with 
other levels of government. 
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This May 1996 Report comprises 12 chapters and a Foreword and Main Points. In 
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the order form are found at the end of this chapter. 
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AUDITOR GENERAL OF CANADA VERIFICATEUR GENERAL DU CANADA 


To The Honourable the Speaker of the House of Commons: 


I have the honour to transmit herewith my first Report of 1996 to the House of 
Commons, to be laid before the House in accordance with the provisions of section 7(5) of 
the Auditor General Act. 


Sa ee 


L. Denis Desautels, FCA 
Auditor General of Canada 


OTTAWA, 7 May 1996 
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Report of the Auditor General 

to the House of Commons 

for May 1996 


Foreword 


Iam pleased to table the May volume of my 1996 Report. Bound with this Foreword are the Main Points and 
Chapter 1 — Other Audit Observations. Eleven additional chapters are issued separately: 


Zz. The Implementation of Federal Environmental Stewardship 

3; Evaluation in the Federal Government 

4, Internal Audit in Departments and Agencies 

Sh The Reform of Classification and Job Evaluation in the Public Service 


Gre Peacekeeping 
— Foreign Affairs and International Trade Canada 
— National Defence 


8. Canadian Security Intelligence Service — National Headquarters Building Project 

9. Agriculture and Agri-Food Canada — Animal and Plant Health: Inspection and Regulation 
10. Correctional Service Canada — Rehabilitation Programs for Offenders 

ike Revenue Canada — Combatting Income Tax Avoidance 


12% Veterans Affairs Canada — Health Care 


The structure of government and the design of government programs are evolving in response to shrinking 
resources and changing public expectations. Several of the chapters in this Report underscore the importance of 
having strong program and departmental leadership, commitment to policy choices and the right kinds of 
information on which to base decisions. The availability of information and commitment to follow good 
management practices will affect this evolution of government and thus, in one way or another, the future for most 
Canadians. 


The fundamental tools of good management are available to government managers. As the chapters in this Report 
show, some are using the tools better and more consistently than in the past. Others have demonstrated a lack of 
commitment to stated policy and have not provided the leadership necessary to demonstrate good program 
management. The Report also notes that there are still gaps in providing the information to Parliament in a way 
that can be used productively. 


Whether programs are obtaining the results they were designed for; whether there are less costly but equally 
effective alternatives; whether limited resources are being targeted at areas where they are needed most or where 
they have the greatest impact; whether policy makers know the implications of the alternatives they choose or 
eliminate: these are the kinds of information that public servants need to obtain, use and provide so that 
Canadians can rely with confidence on their government to make the informed decisions needed to guide them 
into the future. 
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Other Audit Observations 


Chapter 1 — Main Points 


1.1 The Auditor General Act requires the Auditor General to include in his Reports matters of significance 
that, in his opinion, should be brought to the attention of the House of Commons. 


1.2 The “Other Audit Observations” chapter fulfils a special role in the Report. Other chapters normally 
describe the findings of the comprehensive audits we perform in particular departments; or they report on audits 
and studies of issues that relate to operations of the government as a whole. This chapter reports on specific 
matters that have come to our attention during our financial and compliance audits of the Accounts of Canada, 
Crown corporations and other entities, or during our value-for-money audits. 


1.3 The chapter normally contains observations concerning departmental expenditures and revenues. The 
issues addressed generally involve failure to comply with authorities, and the expenditure of money without due 
regard to economy. 


1.4 Although the individual audit observations report matters of significance, they should not be used as a 
basis for drawing conclusions about matters we did not examine. 


15 Observations reported cover the following: 
e the lack of clarity about results expected from a $7.5 million community adjustment initiative; and 


e serious concerns about the administration of the Income Tax Act involving the movement out of 
Canada of at least $2 billion of assets held in family trusts. 
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The Implementation of Federal 


Environmental Stewardship 


Assistant Auditor General: Robert R. Lalonde 
Responsible Auditor: Wayne Cluskey 


Chapter 2 - Main Points 


2.1 Since its announcement in the Green Plan (1990), environmental stewardship has been practised by 
federal departments and agencies. In 1992 the federal Code of Environmental Stewardship provided direction for 
the “greening” of government operations, which includes the preparation of environmental action plans. While 
most agencies will continue to operate under the Code, departments as of December 1995 are required to prepare 
sustainable development strategies and plans for their implementation, following amendments to the Auditor 
General Act. 


2.2 A strategic framework for environmental stewardship, laid out in 1991, assigned responsibilities to 
departments and agencies. However, the program was not implemented as foreseen. This is illustrated by the 


generally poor response rate of departments and agencies in preparing and updating action plans and providing 
progress reports. 


2.3 As the greening process is now ina period of transition, it is a good time to review progress to date and 
make any necessary changes to maintain its momentum. In our view, while considerable progress has been made 
since 1992, there are lessons to be learned from past experiences. 


2.4 There is a need for leadership for the greening process both government-wide and within individual 
departments and agencies. Responsibility for the implementation of the process must be clarified. 


2.5 Information management needs to be improved if the process is to remain on track. While Environment 
Canada, as facilitator, has developed a number of tools including pilot projects, workshops and guidance 
documents, these have not always reached clients or been tailored to their needs. Furthermore, the opportunity has 


not been taken to develop a government-wide data bank to be used as an information source for all participants 
and for aggregate reporting. 


2.6 Aggregate reporting provides decision makers with the information necessary to ensure that the greening 
process is focussed, on schedule and moving toward the goal of sustainable development. In fact, reporting has 


been selective to date and no true aggregate report has been produced, nor do there appear to be any plans for such 
aggregate reporting in the future. 
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Evaluation in the Federal 
Government 


Assistant Auditor General: Maria Barrados 
Responsible Auditor: Stan Divorski 


Chapter 3 — Main Points 


oo Government has recognized the need for better information on program results to support decisions and 
to improve the way government works. Information on the “bottom line” of government programs requires 
disciplined measurement and analysis of program effectiveness. The need for studies of program effectiveness — 
commonly referred to as evaluations — is growing as a result of scrutiny being given to government programs. 


3.2 A changed environment has created challenges for planning and conducting evaluations. A new Treasury 
Board review policy recognizes the responsibility of program managers for measuring program performance. The 
policy also recognizes that effectiveness measurement and analysis can come from a variety of studies, but does 
not make clear the standards that apply when such measurement and analysis is conducted as part of reviews other 
than evaluations. 


ao The choice of evaluation topics in departments reflects decisions based largely on departmental interests 
and priorities. Evaluations continue to emphasize the needs of departmental managers — focussing on smaller 
program components and operational matters. These studies can provide valuable support to management. Good 
government also requires overall program effectiveness information for government decision making and broader 
accountability. This is not consistently provided. Departments have shown that they can respond to demands for 
information on the accomplishments of key programs. For example, we found they have made progress in 
assessing the effectiveness of programs that spend more than $1 billion annually. 


3.4 Departments, with the exception of some smaller ones, have maintained their levels of financial support 
for evaluation; however, certain key aspects of the management of evaluation in departments have not improved. 
The scope and quality of evaluations remain matters for concern. 


ee) Government has introduced measures with the potential to improve its planning of evaluation and to 
support continued progress in evaluating key programs. A renewed expenditure management system requires 
departments to submit business plans with priorities for reviews of their programs, including evaluations and 
internal audits. This approach has the potential to link departmental evaluation plans to the budgeting process. In 
the first year, about two thirds of the plans identified review priorities. However, the system is in the early stages 
of development. The government still needs to ensure that individual departments’ evaluation priorities reflect 
government’s evaluation priorities. In order to accomplish this, the government needs to state its evaluation 
priorities systematically, identify gaps in review activity and ensure that those gaps are filled. 


3.6 The President of Treasury Board has submitted to Parliament a report on the performance of government 
review that is a promising first step. Future reports need to use more concrete measures and provide a more 
adequate picture of overall government success in measuring and analyzing the effectiveness of programs. 
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Internal Audit in Departments and 


Agencies 


Assistant Auditor General: David H. Roth 
Responsible Auditor: Bruce C. Sloan 


Chapter 4 - Main Points 


4.1 The government is going through a period of tremendous change, in which departments are being 
streamlined and restructured in an effort to reduce the administrative cost of government while maintaining or 
improving the delivery of government programs to Canadians. Although these changes may influence the nature 
and location of organizational controls, they have not lessened the need for the organization to be in control. 
Within this environment, the need for effective internal audit has never been more important to ensure that 
departmental programs and operations are properly managed. 


4.2 An effective internal audit unit is a fundamental management tool. It can provide deputy ministers with 
assurances about the efficiency and effectiveness of key financial, administrative and operational activities and 
departmental management practices, along with suggestions for improvement. 


4.3 The support of senior management is essential if internal audit is to be an effective, contributing part of 
an organization. To a great extent, the relationship between internal audit and senior management determines the 
respect internal audit receives from operating managers. We found that, while senior management expressed 
general support for the internal audit function, its perception of the role of internal audit was narrower than that of 
senior management in the private sector. 


4.4 The Treasury Board Secretariat carries out a number of activities that it believes are designed to further 
the implementation and monitor the effectiveness of the internal audit policy, one element of the Review Policy. 
In our view, these efforts have not been sufficient to provide an assessment of the effectiveness of the 
government’s internal audit policy. 


4.5 While some internal audit units have improved their performance since 1993, we found that, overall, 
there has been little improvement since our 1993 audit. Of the seven departmental internal audit units we 
reviewed in detail, four do not meet established standards for the practice of internal audit. 


4.6 We noted that internal audit coverage of computer systems under development has improved. However, 
further improvement is still needed in other areas if internal audit is to fulfill its potential. 


4.7 To fulfill that potential, internal audit must improve the balance of the scope of its work, including both 
compliance and control issues and the assessment of the efficiency and effectiveness of departmental operations. 


4.8 As part of our audit, we assessed the extent to which departments were using appropriate methods and 
techniques for measuring the effectiveness of their internal audit units — that is, their contribution to improving 
the way a department operates. We found that in 1993 virtually no audit units were measuring their own 
performance. However, approximately half of the audit units are now doing so in some way. 
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Job Evaluation in the 
Public Service 


Assistant Auditor General: Maria Barrados 
Responsible Auditor: Jacques Goyer 


Chapter 5 — Main Points 


5.1 The reform of classification and job evaluation was one of the important systemic changes associated 
with public service renewal. Because of important links with other systems such as labour relations and pay 
administration, classification and job evaluation have been considered the “cornerstone” of human resource 
management in the public service. 


5.2 In response to recommendations for change, the Treasury Board Secretariat undertook to develop a single 
job evaluation plan to replace the some 70 currently in use, and to reduce significantly the number of occupational 
groups and sub-groups into which jobs are classified. Given the difficult context, this was quite a challenging task. 


53 Based on our assessment of the June 1995 version of the Universal Classification Standard, our audit 
concluded that, after some six years, the challenge posed by the reform has been only partially met: 


e The new job evaluation plan developed by the Secretariat is a definite improvement and has many 
promising features, but a number of significant issues need to be addressed to ensure that it is a valid 
and reliable tool, capable of evaluating all jobs in the public service while meeting requirements for 
pay equity that stem from the Canadian Human Rights Act. 


e Opportunities exist to simplify further and/or to reduce the costs of classification and job evaluation 
and some of the related systems and practices, such as pay administration. 


5.4 While the Treasury Board Secretariat has undertaken the reform with a commitment to “cost neutrality”, 
we found that the cost and benefits of the reform were not known with any degree of precision. 


5.5 At the conclusion of the audit the Secretariat was taking a number of important measures to address 
issues identified by the audit. The Standard has potential and can be made to work. Success will depend, however, 
on the willingness and ability of the stakeholders to work together to achieve the objectives of the reform at a 
reasonable cost to the taxpayer. 
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The Reform of Classification and 


1] 


Peacekeeping — Foreign Affairs 


and International Trade Canada 


Assistant Auditor General: Richard B. Fadden 
Responsible Auditor: Vinod Sahgal 


Chapter 6 - Main Points 


6.1 Foreign Affairs and International Trade Canada views peacekeeping as central to Canada’s foreign policy 
objective of protecting Canada’s security within a stable global framework. Over the years, Canada has 
participated in most peacekeeping missions. 


6.2 The nature of UN peacekeeping has changed dramatically in the post-Cold War era. Before, the main role 
of the United Nations (UN) was largely to monitor truce agreements between states. Now, the UN is increasingly 
called upon to carry out more complex missions that also involve a peace enforcement role both between and 
within states. As well, a role for civil-military co-ordination and co-operation has emerged. 


6.3 The United Nations continues to be the key vehicle for pursuing Canada’s global security objectives; 
however, it faces significant constraints in its ability to carry out its peacekeeping function effectively: 


e The UN’s management points out that member states collectively have not been prepared to 
adequately equip the organization to meet the additional challenges, as shown by experience in 
Somalia, Rwanda, Bosnia and Haiti. 


e The Department points out that missions with unclear mandates and inadequate resources are 
unlikely to succeed, despite improvements in management practices at UN headquarters. 


e Unlike Canada, a number of members are not paying their dues on time, with the result that 


peacekeeping funds are increasingly being used to defray the cost of the regular budget, which is also 
seriously in arrears. 


6.4 Canada places a high priority on reform of the UN, and has worked on several proposals to improve the 
UN’s effectiveness in peacekeeping. However, significant reforms require consensus of the 185-member General 
Assembly. Consensus on change has proved difficult to find. 


6.5 The overall cost of peacekeeping to Canada, both financial and non-financial, is not aggregated in any 
public document. The financial costs have risen sharply in recent years, as a result of the increase in the number 
and size of missions, from about $47 million in 1991-92 to an estimated $240 million in 1995-96, an increase of 
410 percent. The cost of participation in NATO peacekeeping in Bosnia and Canada’s recent decision to send 
troops to Haiti will add to this increase. 


6.6 The Department has no established procedure to carry out evaluations or “lessons learned” exercises in 


the area of peacekeeping from a foreign policy perspective. This increases the risk of not benefiting fully from the 
experience gained from previous participation. 


6.7 Canada has recovered $236 million of its peacekeeping costs from the UN during the last five years. 
Reimbursements estimated at $92 million were receivable at 3] March 1996. The amount and timing of 
reimbursement recovery is largely dependent on the UN’s cash position, the priority Canada gives to the collection 
function, invoices and supporting documentation and acceptance by the United Nations of amounts billed. 


6.8 Informed decision making on peacekeeping matters would be enhanced by greater transparency and 
accountability. Also, the administration for reimbursements from the UN needs to be strengthened. 


(9 
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Peacekeeping — 
National Defence 


Assistant Auditor General: David Rattray 
Responsible Auditor: Peter Kasurak 


Chapter 7 - Main Points 


TA The Canadian Forces have achieved much in their peacekeeping operations and their missions have been 
marked by individual acts of dedication and bravery. 


dom The Department of National Defence plans for peacekeeping missions in a fluid environment, often under 
severe time constraints. It has experienced gaps in its planning for deployments and rotations to missions. The 
Department, however, has improved its staff system to assess the likelihood of success of the overall mission, to 
assess the capability of the Canadian Forces to meet the requirements of a mission and to estimate the impact of a 
deployment on other commitments. 


the) The Land Forces have retained soldiers in their field force who otherwise would have been cut as part of 
force reductions. This accounts for 1,600 of the 3,000 extra troops allowed by the government for peacekeeping. 
Little other increase has occurred to date. Overall, even in the most highly used categories, personnel have not 
been overtasked for peacekeeping duty. There are still many practical limitations to using the Reserves for 
peacekeeping. 


7.4 Peacekeeping duty in the last few years has revealed serious problems in the Land Force’s ability to 
generate multipurpose forces: 


e The Land Force is not completing its collective training plans. Its most comprehensive training 
exercise was replaced by a much smaller exercise due to peacekeeping-related tasks. Many 
lower-level exercises planned in the last few years did not take place. 


e Significant emergency buys were needed to provide minimum levels of equipment for troops 
deployed for peacekeeping. In 1995, National Defence concluded that long-standing deficiencies in 
its armoured vehicles would not allow troops to perform assigned missions within an acceptable level 
of risk. 


e Departmental plans for medical support show that the Department’s planned field hospital will not 
be enough to support the Land Force’s operational plans. 


ffs! The Department sent $822 million in supplies and equipment to support the peacekeeping missions we 
studied. It recorded $67 million in downward adjustments to stock balances and $13 million of losses. During our 
audit, however, the Department could not find a significant portion of the required documentation, and the 
downward adjustments and losses remain largely unexplained. 
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SECS SSS ESSE SENET NE SESSA SV 
Canadian Security Intelligence 


Service 


National Headquarters Building Project 


Assistant Auditor General: David Rattray 
Responsible Auditor: Reno Cyr 


Chapter 8 - Main Points 


8.1 Since its creation in 1984, the Canadian Security Intelligence Service (CSIS) had needed a headquarters 


facility separate from its predecessor, the former Royal Canadian Mounted Police Security Service. 


8.2 The completion of the new national headquarters building in 1995 has allowed CSIS to consolidate its 
headquarters and Ottawa Regional Office activities from ten separate facilities into one facility. 


8.3 In our opinion, this project was well managed overall, as the national headquarters project was completed 
on time and within budget, and meets the functional needs of its users. 


8.4 There are several success factors or lessons learned in this project that would benefit similar construction 
projects; one of those factors is the key role played by CSIS as the building’s user. 


8.5 We believe that the project team took appropriate steps to ensure that the completed facility provides a 
secure environment for CSIS employees and their activities. 


8.6 The fact that the constructed gross area exceeded the original planned gross area by over 13,000 square 
metres, without exceeding the approved budget, raises concerns about how project budgets are calculated and 
about the effectiveness of Treasury Board control over its project approvals. 


8.7 The redesign of the sixth floor, which was done despite the concerns of the project team and at an 


additional cost of approximately $1.4 million, did not, in our opinion, significantly improve the functionality of 
the building. 


8.8 We also noted contract prepayments made in advance of work, contrary to the Financial Administration 


Act, as well as inadequate wording in the contract documents for the architect, resulting in reduced accountability 
and increased costs. 


8.9 We noted examples where, although no authorities or budgets were exceeded, additional funds could 
have been saved on this project. 
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Agriculture and Agri-Food 
Canada 


1 SA Animal and Plant Health: 
ssistant Auditor General: Don Young : : 
Responsible Auditor: Douglas Timmins Inspection and Regulation 


Chapter 9 - Main Points 


9.1 Canada’s relative freedom from animal diseases and plant pests and its high health standards have 
contributed significantly to our nation’s ability to export annually over $17 billion of agriculture and forestry 
products. International recognition of the health status of our domestic plant and animal resources is necessary to 
maintain access to export markets. 


9.2 The changing global agricultural economy has increased the pressure on the Department to allow 
additional imports, increasing risk to Canada. The Department has responded by assessing the risk of imports and 
by reacting quickly to threats identified. At the same time, the Department has made extensive efforts to realign 
its business approach to health protection, while also responding to significant resource reductions. 


93 The Department has developed a risk assessment process and, as a result, is recognized internationally as 
a leader in this area. The process provides the Department with a scientific basis to assist in making risk 
management decisions. Domestic health threats, regulated diseases, weeds and toxic substances have not been and 
will not be assessed in the foreseeable future, using this analytical tool, unless the emphasis shifts away from 
import trade-related activities. Consequently, the Department lacks sufficient reliable information to rank threats 
based on greatest risk, set priorities on a program-wide basis or demonstrate that it has made the best use of its 
available resources. 


9.4 Program objectives have been expressed in measurable terms. However, management has developed few 
performance indicators or other summary information to allow parliamentarians to understand and assess planned 
and actual performance and departmental action in response to serious outbreaks of diseases and pests. 


9.5 Over 1,100 managers, inspectors and laboratory staff deliver the Animal and Plant Health Program, at a 
cost of more than $100 million a year. While the Department has acted on a number of opportunities for cost 
reduction and cost avoidance, it continues to operate the Program and to plan future activities without 
rationalizing the need for the existing level of resources. 


9.6 Cost recovery has been implemented for some services previously provided without charge and will be 
expanded to others, to offset significant reductions in program funding. Considering the difficulties associated 
with maintaining industry competitiveness and international market access, with identifying beneficiaries and with 
developing a flexible and consultative approach to cost recovery, the Department has made progress over the short 
term. Our main concern is that the amounts currently being recovered fall at least 60 percent short of the full costs 
of delivering the service — service of which industry is the primary beneficiary. 
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Correctional Service Canada 


Rehabilitation Programs for Offenders 


Assistant Auditor General: Maria Barrados 
Responsible Auditor: David Brittain 


Chapter 10 - Main Points 


10.1 Part of the mandate of Correctional Service is to rehabilitate offenders in its care. Since most of the 
14,000 offenders currently incarcerated in federal penitentiaries will eventually be released to the community, 
many argue that this role is the most important part of the Service’s mandate. It is particularly important when one 
considers that, historically, 28 percent of released offenders committed new offences after release. 


10.2 Correctional Service spends approximately seven percent of its $1 billion annual budget on programs 
targeted at factors that contribute to criminal behaviour. Traditional programs, such as education, vocational 
training and prison industries, focus on employability. Since 1988, intervention programs have been the main 
focus of the rehabilitation efforts of Correctional Service. These programs focus on factors such as sexual 
deviancy, substance abuse, and antisocial behaviour. Today, Correctional Service has designed an impressive 
series of intervention programs, some of which have received international recognition. 


10.3 We noted weaknesses in the management of the $34 million spent on intervention programs. The Service 
spends a disproportionate amount of its intervention program resources on two contracts without any assurance 
that the right offenders are being treated or any analysis indicating that the results being achieved justify their 
cost. For instance, 17 percent of the expenditure on sex offender programs is spent to treat 20 offenders, while the 
remaining 83 percent is spent on 1,800 offenders. For substance abuse programs, 16 percent of the expenditure is 
spent to treat 100 offenders, while the remaining 84 percent is spent on 5,000 offenders. Another weakness is that 
the Service has not fully established a continuum of programs to properly support offenders in their transition 


from the institution to the community. The Service is currently able to meet only 65 percent of demand for relapse 
prevention programs for sex offenders in the community. 


10.4 There are also weaknesses in the management of the Service’s traditional programs. CORCAN (the 
prison industry operation) is the Service’s most expensive rehabilitation program, using more than $100 million in 
the past four years. CORCAN jobs cost $13,000 per full-time job in 1994-95, while basic education and 
vocational training cost about $7,500 per full-time student. Furthermore, half of the offenders assigned to 
CORCAN did not necessarily need the skills being offered. Another concern is that over 95 percent of the 
Service’s employability resources are focussed in the institutions, leaving very little to help offenders get a job 
once they return to the community. The Service does not have a coherent strategy to deal with offender 
employability. It cannot make the necessary trade-offs for resource allocation. 


10.5 Correctional Service is committed to reintegration and has been successful in developing a wide range of 
intervention programs. However, there is a lack of attention by senior managers to determining the best match of 
resources to their overall objective of safe reintegration. To achieve this balance, the Service will require better 
information on the cost and benefits of its rehabilitation programs. Without such information, the Service lacks the 
capability to strategically reassess and reallocate its investment in rehabilitation programs. 
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Revenue Canada 


Combatting Income Tax Avoidance 


Assistant Auditor General: Shahid Minto 
Responsible Auditor: Barry Elkin 


Chapter 11 — Main Points 


11.1 The cost of tax avoidance is not known. However, the results of Revenue Canada’s program to combat it 
indicate that avoidance continues to pose a serious threat to the tax base. The Department expects the program to 
produce about $365 million in reassessments in 1995-96. 


11.2 The Department has undertaken certain specific initiatives to combat and deter tax avoidance. The 
operations in the district offices we visited have produced significant tax reassessments. 


11.3 Preventing avoidance can produce significant tax revenue. Moreover, it is crucial to the integrity of the 
self-assessment system. Our report has identified areas where opportunities exist to streamline operations and 
strengthen accountability. 
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SELLS a 
Veterans Affairs Canada 


Health Care 


Assistant Auditor General: Wm. F. Radburn 
Responsible Auditor: John O’Brien 


Chapter 12 — Main Points 


12.1 Canada fulfills its obligation to its war veterans, in part, through programs offered by Veterans Affairs 
Canada. The main funded programs of the Department are pensions, economic support and health care. For the 
last 10 years, health care has been the fastest growing program of Veterans Affairs, increasing in cost by 

149 percent over that period because of jurisdictional issues, expanded eligibility for programs, improved benefits 
and inflation, particularly in the late 1980s. 


12.2 The Department has not yet developed a comprehensive plan to meet the future health care needs of its 
clients for the following reasons: 


e It has limited information on its future health care population. There is a significant population of 
veterans and members of the armed forces who are potential eligible clients and who could be 
eligible for Veterans Affairs health care benefits. 


@ The Department does not have a forecast of the likely changes in future health care needs of its client 
population and the impact of these changes on the Department’s programs. 


12.3 Veterans Affairs is committed to reducing the turnaround time for first-level approval of pension 
applications from an average of 18 months to 9 months, within two years. In future, we intend to audit the 
Department’s success in meeting this goal. 


12.4 There is a long-standing unresolved issue between Veterans Affairs and some provinces over 
responsibility for providing certain health care benefits to income-qualified veterans. The Department has 
identified approximately $50 million that it is paying annually for these health care benefits. 


12.5 In our opinion, the Department has Satisfactory controls over eligibility for health care benefits. 


12.6 The Department does not have national guidelines for the quality of long-term institutional care. Control 
over the cost of delivery in contract institutions is deficient, with weak operating agreements, budgets often 
received after the beginning of the fiscal year and backlogs in conducting operating reviews. 


12.7 In implementing changes designed to improve the efficiency of delivery of the Veterans Independence 
Program, Veterans Affairs did not fully comply with conditions established by Treasury Board that required 
continuation of a post-payment verification system. We found weaknesses in the health care needs assessment 
process, also required by Treasury Board. The Department does not have adequate empirical evidence to 


demonstrate the impact the Program has on recipients’ health or on helping recipients live independently in their 
homes and communities. 


12.8 The Department has saved money by implementing controls for the provision of non-drug services to 
clients. It has also identified the need to improve controls for drug benefits and is in the process of developing a 
plan to implement these controls in 1997. The Department estimates that it could save $7 million annually and 


improve the quality of care by implementing improved controls. In addition, the Department’s method of 
controlling over-the-counter drugs is costly. 
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Chapter 1 


Other Audit Observations 


The work that led to other audit observations was conducted in accordance with 
the legislative mandate, policies and practices of the Office of the Auditor General. 
These policies and practices embrace the standards recommended by the Public 


Sector Accounting and Auditing Board (PSAAB) of the Canadian Institute of 
Chartered Accountants. 
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Other Audit Observations 


Main Points 


Ls The Auditor General Act requires the Auditor General to include in his Reports matters of significance 
that, in his opinion, should be brought to the attention of the House of Commons. 


1.2 The “Other Audit Observations” chapter fulfils a special role in the Report. Other chapters normally 
describe the findings of the comprehensive audits we perform in particular departments; or they report on audits 
and studies of issues that relate to operations of the government as a whole. This chapter reports on specific 
matters that have come to our attention during our financial and compliance audits of the Accounts of Canada, 
Crown corporations and other entities, or during our value-for-money audits. 


1.3 The chapter normally contains observations concerning departmental expenditures and revenues. The 
issues addressed generally involve failure to comply with authorities, and the expenditure of money without due 
regard to economy. 


1.4 Although the individual audit observations report matters of significance, they should not be used as a 
basis for drawing conclusions about matters we did not examine. 


LS Observations reported cover the following: 
e the lack of clarity about results expected from a $7.5 million community adjustment initiative; and 


e serious concerns about the administration of the Income Tax Act involving the movement out of 
Canada of at least $2 billion of assets held in family trusts. 
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Introduction 


1.6 This chapter contains matters of 
significance that are not included 
elsewhere in the Report and that we 
believe should be drawn to the attention of 
the House of Commons. The matters 
reported were noted during our financial 
and compliance audits of the Accounts of 
Canada, Crown corporations and other 
entities, or during our value-for-money 
audits. 


i ber Section 7(2) of the Auditor 
General Act requires the Auditor General 
to call to the attention of the House of 
Commons any significant cases where he 
has observed that: 


e accounts have not been faithfully and 
properly maintained or public money has 
not been fully accounted for or paid, 
where so required by law, into the 
Consolidated Revenue Fund; 


e essential records have not been 
maintained or the rules and procedures 
applied have been insufficient to 
safeguard and control public property; to 
secure an effective check on the 


assessment, collection and proper 
allocation of revenue; and to ensure that 
expenditures have been made only as 
authorized; 


e money has been expended for 
purposes other than those for which it was 
appropriated by Parliament; 


e¢ money has been expended without 
due regard to economy or efficiency; or 


e satisfactory procedures have not 
been established to measure and report the 
effectiveness of programs, where such 
procedures could appropriately and 
reasonably be implemented. 


1.8 Each of the matters of 
significance reported in this chapter was 
examined in accordance with generally 
accepted auditing standards; accordingly, 
our examinations included such tests and 
other procedures as we considered 
necessary in the circumstances. The 
matters reported should not be used as a 
basis for drawing conclusions about 
matters not examined. The instances that 
we have observed are described in this 
chapter under the appropriate department 
headings. 


Other Audit Observations 


This chapter contains 
a number of 
observations on 
matters of significance 
not included 
elsewhere in the 
Report. 
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Other Audit Observations 


Atlantic Canada 
Opportunities Agency 


Assistant Auditor General: Wm. F. Radburn 


Responsible Auditor: John O’Brien 


Concerns about a community adjustment initiative 


maintenance, and capital improvements. 


The Atlantic Canada Opportunities Agency is the federal department responsible 
for adjustment initiatives designed to assist communities affected by the reduction 
of military infrastructure in Atlantic Canada, including the closure of Canadian 
Forces Base Cornwallis in Nova Scotia. The Agency established the Cornwallis 
Park Development Association, a non-profit society, to implement a community 
adjustment initiative. The Agency did not clearly establish the results expected from 
this initiative, nor did it determine, on a timely basis, the activities eligible for 
funding through a $7.5 million contribution agreement with the Association. 
Important conditions were not included in the contribution agreement, such as the 
requirement for the recipient to develop and use conflict-of-interest guidelines in 
the expenditure of funds provided by the Agency, and the definition of specific 
eligible costs. The Agency did not receive a business plan from the Association until 
fourteen months following its creation. During that period, the Agency provided the 
Association with $2.7 million, or 36 percent of its approved funding. The 

| Association spent these funds primarily on facilities administration, operation and 


Background 


1.9 The February 1994 Budget 
announced the closure or reduction of 
seven military establishments in Atlantic 
Canada, including the closure of Canadian 
Forces Base Cornwallis in Nova Scotia. 
As part of the government’s 
multidepartmental strategy for the military 
infrastructure reduction in Atlantic 
Canada, the Atlantic Canada 
Opportunities Agency (the Agency) was 
assigned responsibility for co-ordinating 
federal efforts to support the affected 
communities. This included responsibility 
for establishing measures to assist 
community adjustment efforts. The 
Department of National Defence was 
responsible for co-ordinating the interim 
custodianship of the former base assets 
and was to manage the infrastructure 


disposal process through Public Works 
and Government Services Canada. 


1.10 On 9 June 1994, the Treasury 
Board approved Agency funding of 

$30 million for base closure community 
support in Atlantic Canada. For the 
Canadian Forces Base Cornwallis closure, 
the maximum funding allowed was set at 
$7.5 million. In its approval, the Treasury 
Board attached terms and conditions to 
provide a framework for developing 
contribution arrangements with eligible 
recipients. These terms and conditions 
authorized the Agency to fund local 
development agencies based on detailed 
agreements that specified how funds were 
to be advanced, expended and accounted 
for. 


1.11 On 29 July 1994, the Agency 
established a local development agency, 
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Cornwallis Park Development Association 
(the Association), to provide a focus for 
economic development opportunities for 
the affected communities and a vehicle to 
take legal title and ownership of base 
assets identified as having a potential 
viable reuse. The Association was 
incorporated as a non-profit society in the 
province of Nova Scotia. As provided for 
in the bylaws of the Association, the 
Minister responsible for the Atlantic 
Canada Opportunities Agency appointed 
the Chairman of the Cornwallis Park 
Development Association. 


1.12 On 16 August 1994, the Agency 
entered into a contribution agreement with 
the Association to fund its activities for 
five years, up to a total of $7.5 million. 


1.13 On 30 March 1995, National 
Defence entered into a lease with the 
Association that effectively transferred 
control and responsibility for the former 
base and all remaining assets to the 
Association as of 1 April 1995. 


1.14.  On27 April 1995, the Treasury 
Board approved the sale of the former 
base to the Association, at a nominal 
value of $1. The rationale for the sale of 
the assets for $1 was that the market value 
of the fixed assets was less than the cost to 
sell. In addition, National Defence was to 
pay a lump sum grant of $6 million to the 
Association in recognition of the cost to 
demolish base assets having no further 
use. National Defence remained 
responsible for the clean-up of any 
existing adverse environmental 
conditions. The sale was finalized on 

31 January 1996. We have been informed 
by National Defence that the military 
police are investigating concerns about the 
removal of assets from the base prior to 
the transfer of ownership. 


1.15 We conducted an audit of the 
Agency’s involvement in the community 


Other Audit Observations 


adjustment initiative with the objective of 
assessing the adequacy of its controls over 
this initiative. We did not audit the 
activities of the Cornwallis Park 
Development Association. 


Issues 


1.16 The contribution agreement 
between the Agency and the Association 
established the conditions for funding the 
Association’s activities. The agreement 
did not include a requirement for the 
Association to develop and use guidelines 
for avoiding conflict of interest in the 
expenditure of funds provided by the 
Agency. This was a specific condition 
required by the Treasury Board. 


1.17 In addition, the agreement did not 
include the following conditions that we 
would have expected to see: 


e It did not clearly establish indicators 
of economic development results expected 
from the Association’s activities. 


e It did not require the Association to 
prepare a plan or any other document, 
such as an annual budget, that established 
the Association’s strategic direction and 
planned activities. 


e It did not establish a definition of 
specific costs eligible for Agency funding. 
This made it very difficult for the Agency 
to determine which of the Association’s 
expenditures should be funded. 


1.18 In approving Agency funding, the 
Treasury Board specified that the 
Association could not receive funding for 
costs incurred for the management and 
maintenance of base assets. The Agency 
included a clause in the agreement that 
restricted its funding to those costs 
necessary to support assets that were 
economically reusable in the short term. 
However, we noted that Agency funding 
was spent primarily on facilities 
administration, operation and maintenance 
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and capital improvements. Agency 
officials have informed us that the purpose 
of these expenditures was to control 
long-term operating costs and prevent the 
deterioration of certain former base assets. 


1.19 Agency officials have indicated 
to us that the arrangement with the 
Association was designed to give it 
latitude in determining and meeting the 
local economic development needs. 
Payments to the Association were based 
on incurred expenditures and three-month 
forecasts. The Association received 

$2.7 million or 36 percent of its approved 
Agency funding by 15 August 1995. Of 
this amount, $1.6 million was advanced to 
the Association so that it could operate a 
sea cadet summer camp that was 
subsequently fully funded by National 
Defence. The Association incurred 
additional, eligible costs not related to the 
sea cadet camp, which the Agency applied 
against the $1.6 million advance. As of 
30 November 1995, the Association 
reported that it had incurred a total of 
$3.2 million in eligible costs, $0.5 million 
more than it had received from the 
Agency. We are concerned that the 
Agency did not receive an overall strategy 
for the proposed use of its funding until it 
received a business plan on 1 December 
1995 — fourteen months after creation of 
the Association. 


1.20 In late August 1995, after 
receiving a request for an additional 
$2.4 million in funding, the Agency 
conducted a review of the Association’s 
operation. As a result of the review, on 
11 September 1995 the Agency notified 
the Association that it had discontinued 
further funding until the following key 
conditions were met: 


¢ the completion of an acceptable 
business plan; 


e establishment of an acceptable 
division of duties between staff and the 
Board of Directors of the Association; 


¢ development of acceptable written 
operating procedures, approved by the 
Board of Directors and covering, at a 
minimum, the hiring, tendering and 
purchasing practices of staff; and 


e alonger-term financial forecast for 
the disposition of revenue from operations 
and the $6 million grant from National 
Defence. 


1.21 The Association has responded to 
the Agency’s conditions. However, key 
aspects of the plan are still dependent on 
additional studies and planning. For 
example, the business plan identifies the 
need to develop a long-term master plan 
for facilities. This plan is projected to be 
completed by December 1996. 


1.22 The Agency has informed us that, 
at a meeting held on 26 March 1996, the 
Board of Directors of the Association 
approved guidelines for avoiding conflict 
of interest in the expenditure of funds 
provided by the Agency. 


1.23 In January 1996, the Agency 
resumed funding for this initiative, after 
implementing additional controls on the 
processing of claims and provision of 
advances. Since then, $150,000 in 
additional funding has been disbursed by 
the Agency. 


Conclusion 


1.24 The Agency has not established 
clear expectations of what it plans to 
achieve through the funding of the 
Association. Further, the Agency did not 
receive a business plan describing the 
Association’s intentions until $2.7 million 
in Agency funding had been spent. 


1.25 The Agency failed to include 
important conditions in the contribution 
agreement with the Association and did 
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not fully comply with Treasury Board 
guidance in its arrangements with the 
Association. 


Agency’s comments: It is a major 
challenge for a local development group 
to transform a large military base, with 
annual expenditures of over $30 million 
and 565 employees, into new, sustainable 
Jobs for its citizens. The Cornwallis Park 
Development Association (CPDA) took on 
that challenge. This was not an audit of 
the CPDA, and therefore it does not 
address a number of the allegations that 
have been made over past months. As the 
Auditor General points out, the military 
police are investigating concerns about 
the removal of assets from the base prior 
to the transfer of ownership. 
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The CPDA has made considerable 
progress; yet as the Auditor General 
points out, further improvements are 
needed. Already CPDA has secured a 
$2.0 million annual contract to operate 

a sea cadet training facility, has 
restructured the Park’s utilities operation 
to limit operating costs, has leased a 
number of buildings to new tenants and is 
pursuing several large new development 
opportunities. CPDA has prepared a 
long-term business plan and the Nova 
Scotia office of ACOA is working with 
CPDA to strengthen its results-targeting 
capabilities. Strict conflict-of-interest 
guidelines have been adopted by the 
CPDA Board. Further refinements of the 
guidelines for costs eligible for ACOA 
funding are being negotiated by the Nova 
Scotia office and will be incorporated in a 
revised contribution agreement with 
CPDA. 
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Revenue Canada 


Assistant Auditor General: Shahid Minto 
Responsible Auditor: Barry Elkin 


Serious concerns about the administration of the Income Tax Act involving the 
movement out of Canada of at least $2 billion of assets held in family trusts 


the rulings. 


When taxpayers contemplate transactions for which the tax treatment is uncertain, 
they can request an advance ruling from Revenue Canada. Advance rulings 
guarantee the income tax effect of the contemplated transactions. We have observed 
two advance rulings relating to moving at least $2 billion of assets held in family 
trusts into the United States from Canada. In our view, the transactions ruled on 
may have circumvented the intent of the law regarding the taxation of capital gains. 
Therefore, we are concerned that Revenue Canada may have eroded the tax base by 
forfeiting a legitimate future claim to many millions of dollars in tax revenue. As 
well, we are concerned about the lack of documentation and analysis of key 
decisions made by the Department and the potential impact of those decisions. 
Furthermore, because the rulings were not made public until recently, other 
taxpayers may have been denied the benefit made available to those who received 


Background 


1.26 Revenue Canada provides 
advance income tax rulings to facilitate 
voluntary compliance with the Income Tax 
Act, uniformity of application of the law 
and self-assessment by taxpayers. 
Advance income tax rulings provide 
certainty to a taxpayer by guaranteeing the 
income tax effects of specific transactions 
contemplated by the taxpayer. They are 
generally provided in situations where the 
tax treatment is uncertain. 


1.27 The processing of requests for 
advance rulings requires expert analysis of 
tax law and policy. In determining its 
position, Revenue Canada consults at 
times with the Department of Finance to 
seek its views on the tax policy intent 
behind various tax provisions. It also 
consults with the Department of Justice 
for its legal interpretation of the 
provisions. While Revenue Canada may 


seek the views of others, it is responsible 
for administering the law. 


The taxation of capital gains 


1.28 The Canadian income tax system 
is complex, but there are certain basic 
principles on which the system is founded. 
One principle is that income tax is applied 
on the basis of residency. Another 
principle is that there is tax on capital 
gains, a particular form of income from 


property. 


1.29 With certain exceptions, the 
intent of the Income Tax Act is that capital 
gains accrued after 1971 on property held 
by residents of Canada are subject to 
taxation. During the lifetime of a resident 
of Canada, when a property is disposed of 
and a gain is realized the gain is taxed. 
Generally, when a resident of Canada dies, 
gains on property holdings are deemed to 
be realized and are taxed accordingly. 
Gains on certain property holdings are 
also deemed to be realized and are taxed 
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when a person leaves Canada to take up 
residence elsewhere. When the tax system 
is viewed in these simple terms, one can 
see that it should not be possible to escape 
tax on capital gains. 


1.30 For a trust that is resident in 
Canada, when it moves property out of 
Canada the general rule is that capital 
gains accrued since 1971 on that property 
are deemed to be realized and are taxed 
accordingly. An exception is made for a 
class of property known as “taxable 
Canadian property”. The rationale for the 
exception is that Canada still retains its 
right to tax capital gains on that property 
(subject to possible exemption under a tax 
treaty) when those capital gains are 
actually realized, even though the 
property is held by a non-resident. In other 
words, the interests of Canada are still 
protected; the tax base is not eroded. 


1.31 However, only a certain type of 
property owned by a trust is included in 
“taxable Canadian property”. Generally, 
the nature of that type of property is such 
that Canada will be able to enforce its 
claim to any tax revenue. The prime 
example is real estate situated in Canada. 
Private company shares are also classified 
as “taxable Canadian property”. In 
contrast, shares in a public company are 
not generally classified as “taxable 
Canadian property’. The law allows an 
individual and a corporation resident in 
Canada who leave Canada to elect to have 
public company shares become “taxable 
Canadian property” upon giving 
satisfactory security to the Minister of 
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National Revenue. Trusts are specifically 
prohibited from making this election. 


1.32 We have observed two significant 
favourable advance income tax rulings 
relating to moving at least $2 billion of 
assets held in family trusts into the United 
States from Canada. 


1.33 A 1985 advance income tax 
ruling said that public company shares 
could leave Canada on a tax-free basis. 
In 1984, an accounting firm asked for an 
advance income tax ruling on behalf of a 
client. The essence of the ruling it 
requested would be a declaration by 
Revenue Canada that it would consider 
public company shares held by a trust to 
be “taxable Canadian property” and that a 
change in residence of the trust, therefore, 
would not cause a deemed realization of 
capital gains on the shares. 


1.34 Revenue Canada ruled that the 
public company shares held by the trust 
could leave Canada on a tax-free basis 
because they would be considered 
“taxable Canadian property”. The ruling, 
which was issued in January 1985, is 
shown in Exhibit 1.1. 


135 A 1985 income tax opinion on a 
similar case said that public company 
shares would be subject to tax on 
leaving Canada. One week after issuing 
the 1985 advance income tax ruling, 
Revenue Canada received a request for an 
opinion (an opinion is not binding on 
Revenue Canada, whereas a ruling is) 
from another accounting firm, in a case 
where the circumstances were 
comparable. In May 1985, an opinion was 


Our ruling is that the trust will not be deemed by subsection 48(1) of the Act to dispose of its 
shares in the public corporations which it acquired as a result of an exchange of its ownership 
in shares of a private corporation when it ceases to be resident in Canada because those shares 


Exhibit 1.1 


The Technical Wording of the 
1985 Advance Income Tax 


will constitute taxable Canadian property within paragraph 85(1)(i) and subparagraph Ruling 
115(1)(b)Gii) of the Act. 
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Exhibit 1.2 


The 1985 Income Tax Opinion 


issued to the effect that public company 
shares could not be considered “taxable 
Canadian property” to a resident of 
Canada. The opinion and the ruling were 
both issued by the Non-Corporate Rulings 
Division. The opinion is shown in 

Exhibit 1.2. 


1.36 We attempted to review the 
analysis supporting the opinion. We were 
informed that the file contains only the 
request for the opinion and the opinion 
itself. There is nothing in the file to 
support the reasoning behind the opinion. 
Revenue Canada advised us that it now 
considers the opinion to be technically 
incorrect. At the completion of our audit, 
the accounting firm had not been advised 
of this. 


E37 A 1991 advance income tax 
ruling again said that public company 
shares could cross the border ona 
tax-free basis. In 1991, Revenue Canada 


issued an advance income tax ruling to a 
legal firm on behalf of another trust 
related to the parties who had received the 
1985 advance income tax ruling. While 
some of the details are different, the main 
issues are identical to those in the 1985 
advance income tax ruling. 


1.38 The 1991 ruling dealt with the 
income tax consequences for a trust that 
would cease to be resident in Canada and 
would become resident in the United 
States (this trust is referred to as 
“Protective Trust” in the ruling). Shortly 
after it became resident in the United 
States it would receive public company 
shares from a trust resident in Canada (this 
trust is referred to as “Family Trust” in the 
ruling). In this instance, however, the trust 
leaving Canada had been in existence for 
less than 10 years. Because of a provision 
in the tax treaty between Canada and the 
United States, this meant Canada would 
have no right to tax the accrued capital 


incorporation). 


You request our views with respect to the meaning of “taxable Canadian property” in the 
context of a distribution by a resident trust to a non-resident beneficiary of shares which were 
acquired by the trust in an exchange to which section 85.1 of the Act applied. You suggest 
that, if the particular shares were acquired in exchange for shares which represented 25 
percent or more of the issued shares of a class of capital stock of a “public corporation” such 
shares would constitute taxable Canadian property of the trust by virtue of paragraph 85.1(1) 
and subparagraph 115(1)(b)(ix) of the Act. 


We do not agree. You point out, the general scheme of the Act contemplates the ownership 
of taxable Canadian property with reference to non—residents (e.g. in paragraph 48(1)(a) of 
the Act which refers to “... any property ... that would be taxable Canadian property if at no 
time in the year he had been resident in Canada ...”). Similar wording is found in subsection 
107(5), and subparagraph 108(1)(d.1)(ii) of the Act. In this context, we do not agree that 
paragraph 133(1)(c) of the Act contemplates the ownership of taxable Canadian property by 
a resident of Canada. It clearly contemplates two situations; the one to which you refer 
contemplates a corporation not resident in Canada (presumably a pre—April 26, 1985 


The provisions of subsection 107(5) of the Act, like the provisions of subsection 48(1) of the 
Act, permit the deferral of the recognition of the gain or loss on the disposition (or deemed 
disposition) in respect of property that, when ultimately disposed of, will normally be 
recognized for Canadian tax purposes (subject to applicable treaty protection). In this regard, 
| We are therefore also of the view that the shares, which you describe, would not be 
considered taxable Canadian property in the hands of the non-resident beneficiary merely 
because they were acquired by the trust as described above. 
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gain on the shares that moved to the 
United States when that gain is realized by 
the Protective Trust. 


1.39 In giving a favourable ruling, 
Revenue Canada accepted the taxpayer’s 
representation that the public company 
shares held by the Canadian Family Trust 
constituted “taxable Canadian property”. 
This representation was based on an 
earlier transaction in which the trust had 
received the public company shares in 
exchange for private company shares. 


1.40 Before issuing the ruling, 
Revenue Canada was fully aware of the 
risk (outlined in paragraph 1.38) presented 
by the then-proposed transaction. The 
advance ruling was issued subject to the 
condition that the taxpayer provide an 
undertaking and a waiver (see 

Exhibit 1.3). 


1.41 The trust moving to the United 
States (Protective Trust) provided an 
undertaking that if it sold the shares 
within 10 years from the date it became a 
resident of the United States, it would not 
invoke the provision in the Canada-U.S. 
tax treaty that could enable it to escape 
Canadian income tax completely on the 
capital gain accrued to the date it left 
Canada. 


1.42 The Family Trust provided a 
waiver effective for 10 years that would 
allow Revenue Canada to reassess it. 
Revenue Canada’s files and the waiver 
show that if the Department exercised the 
waiver it would have to reverse its 
previous position and issue a reassessment 
on the basis that the public company 
shares were not “taxable Canadian 
property” and that the accrued capital gain 
should therefore have been subject to tax 
in Canada when the shares left in 1991. 


1.43 The 1991 advance income tax 
ruling is shown in Exhibit 1.3; Exhibit 1.4 
is a chronology of the key events leading 
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up to the issuance of the ruling; and 
Exhibit 1.5 shows the transactions entered 
into by the taxpayer. 


issues 


The transactions may have 
circumvented the intent of the law 


1.44 As we have noted, the general 
rule for a trust resident in Canada is that 
when it moves property out of Canada 
there will be a deemed realization of 
capital gains accrued since 1971 on that 
property. An exception is made for a class 
of property known as “taxable Canadian 
property”. Generally, public company 
shares are not considered “taxable 
Canadian property”. 


1.45 Some years before the 1991 
ruling, the Family Trust had received the 
public company shares in exchange for 
private company shares. At the time of the 
exchange, both trusts were resident in 
Canada. The section in the Income Tax Act 
(paragraph 85(1)(i)) that characterizes 
public company shares as “taxable 
Canadian property” when they are 
exchanged for private company shares 
clearly applies to non-residents. Revenue 
Canada had to decide whether this 
provision applied to residents as well. In 
the end, supported by advice from the 
Department of Finance, Revenue Canada’s 
view was that residents could hold 
“taxable Canadian property”. 


1.46 
paragraph in the Income Tax Act, 

paragraph 97(2)(c). That paragraph, 
dealing with Canadian partnerships, 


This advice was based on one 


parallels other rollover provisions in the 
Act but its purpose appears unclear. Read 
literally, and in context with section 102 
of the Act, however, the paragraph 
suggests that a Canadian resident can have 
“taxable Canadian property”. At the same 
time, there are several other provisions in 
the Act that strongly suggest that the 


a 
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Exhibit 1.3 


The Technical Wording of the 
1991 Advance Income Tax 
Ruling 


Although the ruling does not 
state that the public company 
shares are “taxable Canadian 
property”, Revenue Canada 
would not have given the 
ruling had it not agreed with 
this. As in the 1985 ruling, the 
underlying assumption is that 
because the public company 
shares were received in 
exchange for private company 
shares, the public company 
shares are “taxable Canadian 
property”. 


overall scheme of the Act contemplates 
the ownership of “taxable Canadian 
property” only by non-residents: for 
example, subsections 107(5) and PESCL) 
and section 128.1. As well, the paragraph 
in the Act that characterizes public 
company shares as “taxable Canadian 
property” when they are exchanged for 
private company shares (paragraph 
85(1)(i)) was intended to prevent a 
non-resident from avoiding Canadian tax. 


1.47 We note that the 1985 income tax 
opinion (Exhibit 1.2) makes a convincing 
argument that the general scheme of the 
Act contemplates the ownership of 
“taxable Canadian property” only by 
non-residents. In our view, the 
transactions ruled on in 1991 may have 
circumvented that intent by characterizing 
public company shares held by a resident 
of Canada as “taxable Canadian property” 


and thereby avoiding tax when those 
shares crossed the border. Furthermore, 
we believe the transactions circumvent the 
overall intent of taxing the accrued capital 
gains on property, other than “taxable 
Canadian property”, when that property 
leaves Canada. 


The ruling may have forfeited a future 
claim to significant tax revenue 


1.48 Documents contained in Revenue 
Canada’s files indicate that, over the two 
months during which the ruling request 
was considered, officials concluded 
repeatedly that the Department should not 
rule favourably. 


1.49 However, during a series of 
meetings on 23 December 1991, senior 
officials of Revenue Canada authorized a 
favourable ruling. Revenue Canada 
advised us that its files do not contain any 
minutes of these meetings or additional 


Provided that: 


® Family Trust provides the District Office of Revenue Canada with a valid waiver 


Our rulings are as follows: 


interest in Family Trust when it ceases to be resident in Canada. 


pursuant to subparagraph 152(4)(a)(ii) (a copy of which will also be provided to the 
Rulings Directorate), effective for ten years after the taxation year in which the shares 
of Public Co. are distributed by Family Trust to Protective Trust, with respect to the 
application of subsection 107(5) in the taxation year of Family Trust in which the 
distribution is made and said waiver is not revoked within the ten—year period, 


Protective Trust provides an undertaking that for a period of 120 months 
commencing on the date Protective Trust becomes a resident of the United States, 
Protective Trust shall not invoke protection under the Canada—United States Tax 
Convention (1980) (the Treaty) regarding any accrued capital gains on the 
distributed Public Co. shares, namely, the provisions of Article XIII(4) of the Treaty, 
and consequently no claim will be made under paragraphs 110(1)(£) and 115(1)(d) of 
the Income Tax Act, in respect of any disposition (including a deemed disposition) of 
the distributed Public Co. shares, 


We confirm that Family Trust will be deemed to dispose of the distributed Public Co. 
shares for an amount determined in accordance with paragraph 107(2)(a) and 
Protective Trust will be deemed to have acquired such shares at a cost determined 
pursuant to paragraph 107(2)(b). 


Subsection 107(5) will not be applicable to the distribution of the Public Co. shares 
by Family Trust to Protective Trust. 


Protective Trust will not be deemed by subsection 48(1) to have disposed of its 
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1991 
March 8 


November 7 
November 25 


December 3 


December 6 


December 12 
December 12 


December 16 


December 18 


December 18 


December 19 


December 20 


December 23 


December 23 


Request for ruling is received. The purposes of the proposed transactions 
are to avoid the deemed disposition at fair market value under the 21-year 
rule and to avoid a deemed disposition on the death of a beneficiary. This 
request is withdrawn and replaced with a ruling requested on 7 November 
1991, as amended. 


Request for ruling is received. 


The taxpayer’s representative provides Revenue Canada with a copy of the 
1985 advance income tax ruling referred to in Exhibit 1.1. 


Revenue Canada advises Finance of the proposed transactions and that it 
intends to refuse to grant a favourable ruling and, if necessary, to bring the 
matter before the general anti-avoidance rule committee to recommend that 
subsection 245(2) be applied to the transaction. 


Revenue Canada and Department of Finance officials meet to discuss the 
issues raised in the 3 December letter. No recommendation is made about 
what should be done. 


Revenue Canada’s rulings review committee decides that a favourable 
ruling should not be provided. 


Revenue Canada advises Finance that it will not rule and requests Finance 
to leave the interpretation of the Act to it. 


The taxpayer’s representatives are advised that a favourable ruling will not 
be provided. They offer to provide a waiver and a statement that they will 
not dispose of the shares for five years. 


Revenue Canada informs the taxpayer’s representatives that it cannot accept 
their offer of 16 December 1991. The representatives offer to not use the 
provisions in the tax treaty between the United States and Canada that 
would allow the taxpayer to escape Canadian tax completely. 


A memorandum is prepared for the Assistant Deputy Minister of Revenue 
Canada advising the Deputy Minister that the Department is unable to rule 
favourably. (We asked Revenue Canada when the memorandum was sent 
to the Deputy Minister and when he was first advised of the ruling request. 
We were advised that there is no information in the file as to when the 
Deputy Minister was first advised.) 


Draft legal opinion is received. 


Senior officials decide that a memorandum prepared for the Deputy 
Minister should be revised. 


A revised memorandum is prepared for the Deputy Minister advising that 
the taxpayer’s proposal is not acceptable because it is the Department’s 
view that when the public company shares are transferred out of Canada, 
tax would be triggered. The letter suggests possible alternatives to the 
taxpayer’s proposal. 


Senior officials meet twice. (There are no minutes of the meetings in the 

file. We asked Revenue Canada to provide us with details of the meetings. 
Revenue Canada advised us that it had examined its files and did not find 
any additional information or details about the meetings.) 


Other Audit Observations 


Exhibit 1.4 


Chronology of the Key Events 
Leading up to the Issuance of 
the 1991 Advance Income Tax 

Ruling 
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Exhibit 1.4 (Cont’d) December 23 


December 23 


December 23 


December 23 


Senior officials meet with Department of Finance officials. (There are no 
minutes of the meeting in the file. We asked Revenue Canada and the 
Department of Finance to provide us with details of the meeting. Both 
departments advised us that they had examined their files and did not 
find any additional information or details about the meeting.) 


Senior officials meet again. (There are no minutes of the meeting in the 
file. We asked Revenue Canada to provide us with details of the 
meeting. Revenue Canada advised us that it had examined its files and 
did not find any additional information or details about the meeting.) 


The rulings officer is advised that the decision has been made to accept 
an undertaking and a waiver and to rule favourably. This is 
notwithstanding the fact that Revenue Canada is aware, based on court 
cases, that there is some uncertainty about the validity and enforceability 
of the undertaking. Revenue Canada decides that the taxpayer would not 
renege on the agreement because doing so would hurt its position in 
future dealings with the Government of Canada. 


Revenue Canada receives a letter from the Department of Finance 
discussing the policy intent of certain provisions in the law crucial to the 
case at hand. These provisions deal with “taxable Canadian property”. 
The letter concludes that in policy terms, residents may own “taxable 
Canadian property”. The letter is used to provide Revenue Canada with 


December 23 


December 24 


1992 


January 13 


some comfort to support its decision to rule favourably. 


The taxpayer’s representatives are advised that the Department will 
accept an undertaking and a waiver and rule favourably. 


The ruling is issued and the waiver and undertaking are received. 


Legal opinion is received. 


details about them. The Department of 
Finance advised us that, for the meeting it 
attended on 23 December, its files also do 
not contain any minutes of, or additional 
details about, the meeting. We note that 
the senior Revenue Canada official 
responsible for rulings did not attend the 
meetings. Some analysis was prepared 
after the fact to justify the ruling. 


1.50 The analysis supporting Revenue 
Canada’s interpretation of the law was 
based on a letter dated 23 December 1991 
from the Department of Finance. Finance 
told us that it was unable to locate any 
documentation to support the conclusions 
reached in the letter or on their potential 


fiscal impact. The letter concluded that 
the law intends that both resident and 
non-resident taxpayers can hold “taxable 
Canadian property”. As we have noted, 
based on this advice Revenue Canada 
concluded that the public company shares 
were “taxable Canadian property”. 


1.51 Whether or not the particular 
shares ought to be considered “taxable 
Canadian property” was ambiguous, due — 
to the complex nature of the Income Tax 
Act provisions where the term is used. 
Also, as we have noted, if Revenue 
Canada acted on the waiver it would have 
to try to sustain a reassessment on the 
basis that the shares were not “taxable 
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Beneficiary 


Beneficial Interest 


Family Trust 


(Assets include 
public company 
shares) 


Beneficiary 


Beneficial Interest 


Protective 
Trust 


Beneficial Interest 


Family Trust 


(Assets include 
public company 
shares) 
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Exhibit 1.5a 


Transactions Entered into by the 
Taxpayer — Original Structure 


Exhibit 1.5b 


Transactions Entered into by the 
Taxpayer — Step 1 


Completed prior to the request for 
the 1991 advance income tax 
ruling. 


The Beneficiary’s interest in the 
Family Trust is transferred on a 
tax-free basis to the Protective 
Trust, in exchange for a beneficial 
interest in it. 
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Exhibit 1.5¢ 


Beneficiary 
Transactions Entered into by the 
Taxpayer —- Step 2 


Beneficial Interest 


Protective 
Trust 


Beneficial Interest CANADA 


Family Trust 


(Assets include 
public company 


The Beneficiary and the Protective shares) 


Trust move to the United States 


Exhibit 1.5d 


Beneficiary 


Transactions Entered into by the 
Taxpayer — Step 3 Beneficial Interest 


Protective Trust 


(Assets now 


ya 
include public 
company shares) 
Public 
Company 
Shares 
Beneficial Interest 
A CANADA 
= Family 
Family Trust distributes public Trust 
company shares to the Protective 
Trust 
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Canadian property”. The issue can be 
settled only by the courts, should a 
relevant case some day be litigated. 


1.52 By accepting the taxpayer’s 
representation that the shares were 
“taxable Canadian property”, Revenue 
Canada may have guaranteed the 
complete elimination of Canadian income 
tax in the future on capital gains on 
property of enormous value. It is unlikely 
that, without the ruling, the proposed 
transactions would have proceeded and 
triggered an immediate tax. However, we 
are concerned that Revenue Canada may 
have eroded the tax base by forfeiting a 
legitimate future claim to hundreds of 
millions of dollars in tax revenue. 


1.53 Furthermore, we would have 
expected Revenue Canada’s files to 
contain minutes of the meeting held with 
the Department of Finance officials and 
some analysis of why Revenue Canada 
officials accepted the advice given. This is 
particularly important in this situation, 
given the amounts involved and the fact 
that the advice received from the 
Department of Finance caused Revenue 
Canada to change its position. We also 
would have expected Revenue Canada’s 
files to contain at least some analysis of 
the potential impact on other sections of 
the Income Tax Act as a result of accepting 
the Department of Finance’s advice. We 
believe that without a documented 
appropriate analysis, public accountability 
for these types of decisions is 
compromised. 


The advance ruling was applied to a 
completed transaction 


1.54 The 1991 ruling was based on the 
view that, because the public company 
shares were acquired in exchange for 
private company shares, they were 
“taxable Canadian property”. This was 
one of the key issues to be resolved before 
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the ruling was issued. However, the 
exchanges were completed transactions. 
While the ruling technically was not given 
on whether the shares were “taxable 
Canadian property’, that was the essence 
of the ruling. Minutes of meetings and 
deliberations of rulings officials, the 
undertaking, the decision of the rulings 
review committee and the memorandum 
to the Deputy Minister all confirm this. 


1.55 Revenue Canada’s Information 
Circular 70-6R2 states that “advance 
income tax rulings will be given only on 
proposed transactions”. Completed 
transactions are normally dealt with by 
audit. Although audit officials had not 
reviewed the exchange transaction to 
determine if the public company shares 
were “taxable Canadian property”, rulings 
officials concluded that they were. 


The undertaking and the waiver have 
limited value 


1.56 Revenue Canada’s files show it 
was concerned that the taxpayer might 
undertake further undisclosed transactions, 
particularly ones designed to avoid tax. To 
address this concern, the taxpayer 
provided an undertaking and a waiver as a 
condition of receiving the 1991 advance 
income tax ruling. 


1.57 Revenue Canada advised us that 
the undertaking given, together with the 
waiver, reinforced the fact that the 
proposed transactions were not designed 
to avoid tax. The Department stated that 
this resolved any concerns about future 
transactions where the application of the 
general anti-avoidance rule could be 
considered, and provided it with a 
measure of protection should the facts 
change in the future. 


1.58 We agree that the undertaking 
and the waiver provided the Department 
with some comfort to support the 
favourable ruling. However, the scope of 
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the waiver was limited and would permit 
Revenue Canada to reassess only on the 
basis that the public company shares were 
not “taxable Canadian property”. But the 
ruling accepted the argument that the 
shares were indeed “taxable Canadian 
property” and could therefore move to the 
United States on a tax-free basis. To act 
subsequently on the waiver, Revenue 
Canada would have to try to sustain a 
reassessment on the basis that the shares 
were not “taxable Canadian property”. If 
Revenue Canada was prepared to use that 
argument in the future if necessary, it must 
have considered the argument valid when 
it issued the ruling. 


1.59 Furthermore, Revenue Canada 
knew that the undertaking provided little 
protection, since it was aware of relevant 
Jurisprudence that had ruled such 
documents unenforceable. The 
memorandum prepared for the Deputy 
Minister also advises that an undertaking 
would not be enforceable if the taxpayer 
later reversed its position and claimed 
protection under the Canada-U.S. tax 
treaty. The documents show that the 
Department decided the taxpayer would 
not renege on the agreement because 
doing so would hurt its position in future 
dealings with the Government of Canada. 
We also note that for the undertaking to 
have any value, the trust would have to 
believe that Revenue Canada could 
convince a court that the public company 
shares were not “taxable Canadian 
property”. The waiver and the undertaking 
were not supported with security. 


Other taxpayers may have been denied 
the benefit made available to those who 
received the rulings 


1.60 The legal firm that requested the 
1991 advance income tax ruling had a 
copy of the 1985 advance income tax 
ruling that had been given to an 
accounting firm. This provided it with an 


advantage over others who were not aware 
of the 1985 advance income tax ruling. 
The accounting firm that received the 
opposite income tax opinion in 1985 may 
not have been aware of the advance ruling 
that same year and may not now be aware 
of the 1991 ruling. We also note that the 
1985 opinion is in the public domain. 


1.61 In 1993 we recommended that 
Revenue Canada release, in severed form, 
issued advance income tax rulings. 
Revenue Canada has released some 
rulings, although it chose not to release 
the rulings discussed in this observation at 
the time they were issued. The 1991 ruling 
was issued in severed form, in 

March 1996. 


1.62 In November 1995, during the 
course of our work on this observation, 
Revenue Canada announced that it plans 
to release all rulings in severed form. We 
support this position. Advance income tax 
rulings are an important part of the tax 
administration process. Making them 
available should result in a more 
consistent and uniform application of the 
law among taxpayers and should improve 
the transparency of the tax system. 
Transparency not only assists taxpayers 
and their advisors but also helps 
parliamentarians and the public 
understand how tax law is used and how it 
works. Transparency should also improve 
accountability. 


Conclusion 


1.63 This audit observation raises 
serious concerns about the administration 
of the Income Tax Act. Advance rulings 
are an important part of the tax 
administration process. They provide 
certainty to a taxpayer by guaranteeing the 
income tax effect of contemplated 
transactions. The Act intends that accrued 
capital gains on property, other than 
“taxable Canadian property”, are taxed 
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when that property leaves Canada. In our 
view, the transactions ruled on in 1991 
may have circumvented that intent. 
Therefore, we are concerned that in giving 
a favourable ruling on the transactions, 
Revenue Canada may have eroded the tax 
base by forfeiting a legitimate future 
claim to many millions of dollars in tax 
revenue. As well, we are concerned about 
the lack of documentation and analysis of 
key decisions made by the Department 
and the potential impact of those 
decisions. Finally, because the rulings 
were not made public until recently, other 
taxpayers may have been denied the 
benefit made available to those who 
received the rulings. 


Revenue Canada’s comments: An 
advance income tax ruling is an assurance 
given by Revenue Canada that 
transactions that will be undertaken by 
taxpayers will have specified tax 
consequences. Taxpayers request advance 
rulings where there is uncertainty about 
the tax consequences of proposed 
transactions that frequently involve 
complex business arrangements with 
significant tax implications. Taxpayers 
require certainty in order to proceed with 
legitimate business transactions. In 
granting an advance ruling, Revenue 
Canada must make decisions with respect 
to the application of law that may be 
ambiguous. 


The granting of a ruling does not preclude 
an audit of the transactions proposed in 
the ruling at a later time. If an audit 
verifies that the facts and proposed 
transactions that were represented in the 
ruling are complete, accurate and are 
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carried out as proposed, Revenue Canada 
would abide by the ruling. 


In interpreting the law that applies to the 
transactions to be undertaken, Revenue 
Canada determines the intent of the law 
by considering not only the words of the 
particular provision, but also the context 
in which the particular provision is found. 


The crux of the rulings is the nature of the 
shares of the public company owned by 
the trusts. If they are taxable Canadian 
property to the Canadian resident trusts, 
they can be distributed by the trusts to the 
non-resident beneficiary without being 
subject to capital gains tax at that time. 
These public company shares were 
acquired by the trusts in exchange for 
shares of a private company owned by the 
trusts. A specific provision of the Income 
Tax Act characterizes public company 
shares as taxable Canadian property when 
they are exchanged for private company 
shares. Whether this provision applies to 
the transactions proposed in the ruling 
depends on whether a resident of Canada 
can own taxable Canadian property. 


As noted by the Auditor General, this 
issue is complex and involved ambiguous 
provisions of the law. As is customary in 
resolving issues of this nature, Revenue 
Canada analyzed the relevant provisions 
of the Income Tax Act and their context 
and consulted the Department of Finance 
to ensure that its interpretation was 
consistent with the intent of the law. On 
the basis of its analysis and advice from 
the Department of Finance, Revenue 
Canada concluded that the public 
company shares were taxable Canadian 
property to the Canadian resident trust. 


This interpretation has been published in 
the form of an advance income tax ruling 
and a general technical interpretation, 
which have been released to the public. 


Report of the Auditor General of Canada — May 1996 1-23 


emp, hi 


5p Seay 
w oy Aan ) ay 


‘2 22 EAT. «% 


oe ee es 


MARIS Wer, / mts MON? el Sone taht ENO ae 


ae 
é - we 
piers sy hi " 
ia i 1 ie 
pits att ell 
a ah 7 
‘ aa 
od P 
, 
ol 
vorr"y , 
Opt giants heat a 
val Re ES 
haath © “eti-winwnl ther 
rant 1h pra AA! nequepalelinen 


oh, oder al . 

Sr er aT Capes 
usd 9g Leeman Ai nmeY caesdind & Ae 
ba poten ge asap oonery ans 
> or Ph clk peep ty canon ait taaRs 
LV *LSbiney hoa pak 
yu So ee 

> ) a0 Me een ie 

ir ie 1) WG sew vy 

oh aiihiey pret. os i 

3 pended check ars 
tay oo alti erdgesiinem acta Fy 
ete ' a 


+} = 


Te ee ee sphinarg sll 
Wy ©: $0 Mites 
Vee Gof ont-ial odeatta 


Bae crn rT a RAS Y 
J eve’ £4.¢ ee : 


was eo net 


a ed 


ae 


Report of the 
Auditor General of Canada to the House of Commons — 1996 


Table of Contents 


Volume 1 - May 1996 


Chapter 
Foreword and Main Points 
1 Other Audit Observations 
i The Implementation of Federal Environmental Stewardship 
i) Evaluation in the Federal Government 
+ Internal Audit in Departments and Agencies 
=) The Reform of Classification and Job Evaluation in the Public Service 
Peacekeeping 
Preface 
Foreign Affairs and International Trade Canada 
7 National Defence 
8 Canadian Security Intelligence Service — National Headquarters Building Project 
2 Agriculture and Agri-Food Canada — Animal and Plant Health: 
Inspection and Regulation 
10 Correctional Service Canada — Rehabilitation Programs for Offenders 
11 Revenue Canada — Combatting Income Tax Avoidance 


2) Veterans Affairs Canada — Health Care 


x lon 


a 


ott jo Hogah 


sheng? ‘np irene 1eHibuA 


grained toolset 
ey; yal — t omnio¥ 


yee? 


Order Form for 1996 Report Material 


If you wish to obtain other 1996 Report material, consult the Main Table of Contents and indicate the chapters and 
formats you wish to receive. Mail the form to: 


Office of the Auditor General of Canada 


240 Sparks Street, Stop 10-1 
Ottawa,ON K1A 0G6 


or FAX form to: (613) 954-0696 


Please Print 


Name: 


Organization: 


Address: 


City: Province: 


Country: Postal Code: 


Telephone: _ ( ) = 


Please Specify Chapter Number and Quantity 


Chapter(s) # ______ English _____ French 
# ______ English _____ French 
# ______ English _____ French 
# Lae EDisn _____ French 
# =) English ___ French 
# _____ English ____ French 


Foreword and Main Points (May 1996) 
Chapter 1 Bilingual 


Diskette (3 1/2” HD) 
(May 1996) English French 


CD-ROM (contains Reports 
from 1987 to May 1996) Bilingual 


Video (contains Selected 
Highlights of the 
Report - May 1996) English French 


The Report is also available on the /nternet at www.oag-bvg.gc.ca 


29°06'baqg—Beo'mmm eb jausajuj Ans ajqiuodsip juawajebe ysa poddey a7 


sie|bue ua siedued} Ud (9661 leW - yoddey np 
sjueyjles sjuiod 
9p XIOYD) ayessedoapiA 


anBuljiq UoIsIaA (9661 JEW 8 L861 ap 
syoddeJ) NOY-9 


siejHue ua siedued} ud (9661 !eIN) 
(GH ..2/} €) eHenbsiq 


anBulliq uoIsIa@A | auideyy 
(9661 !eIN) SjuelIes syulog ya Sodoid-jueay 


sreibueue ~~ —Ssiedueljua Pia! 
sebueua —ssiedueyjuo SOS eer 
sebueua ~ —ssveuedjua = Sanu 
sebueua ~~ —«sedueujua SS =F 
sejbueua ~~ —Ssiedueyjyua SS or! 
sieibue ua ——«ssedureujuua SOS ou (s)aujideug 


‘aayeynos ayjuenb ey J@ (s)eaideyo (sap) np o1ewinu af Jasioaud zayINe) 


= asl ) : auoydgyay 


: Je}sod apo : sheq 
: QOUIAOId : SIIlA 
: 8SSaIpY 


: uonesiuebiC 


: WON 


(eaWwdwI,p sasajoeseo ua) 


9690-186 (€19) ° o4awnu np snaidosajay avd saxkoaua,] NO 


990 VIM (o11e}UQ) EMeNO 
I-07 9418 ‘sysedg ana ‘Qp7 
Bpeuey) Np [e.19u9s In|}eIYLIVA Np nesing 
: / ajupains 
ASSAAPD,] Y 4akOAUA,] Ja UOg af Aduad ‘ajpdiouisd sasaijpu SOP 2]qQUI VI 4AIjNsuod Zayjinaa ‘apupuuod ano] Anog 


SPUBLULWOS ap Ug — 966] ap Woddey 


aa ytd jesnncin Ceitiet: ein (0 vansnn 


ne COLLECEICAS: hes tastes pe egtegyntatens ee gee qeprrtintuse 


ec = foe j 


i abeaaaer: revbécryon <f reyes 
oe eae P Bunterrrow Gry AGH QR e HF 


iw 


1a 
— splays 7. qeute eri ccc 
© laetaonora Pom ‘ee bhai a [3 Ceetluciiage |. wuNseit prs 
Af . 
Be i. page enous | 

ie * yo st ore cumetios pr t Use i 
Ty eee 

* ot ps amnepargon a leap 
a a 

z Seapine tw i ih Oj. SAO ee booras Gane ty yaiegiod Digayatite 


"pr Weatennow nie Guy poe UNUIIGE? Cy ey CtRoneniey 


rn Paella on Stasueuien ner, * 


bag’ 
ts 


© panne ens 1 Pannen netuneicarsrs sn Ba npuieecirn ie ma 


= _aphites ermaasi ye Ries 
: - yantr-fientiue ox jyntain Say Fige 


ee de be leaps sang puted rans Sat 
“SS ipatin; 2 gt FOE EDS a ieee 9 SE : 
ra YOM earl Kew _ery . IGP Ray: op so ih eel z 


<ieacperangly VW wpuri< outed ariagihe: yp ts maint wows aisles 
ea 
cagetaliaatily ion ns sae, | 


; “he Rpm ps2 ans - jp emrdtione: «i ginsisy 
mai ive lage Pet 8 FH sy HEED, | 
nu beue an cous gas go eo 900 


PUES OD Ye ty. deye Pall 


Daten et ee nee aa) qm 26 0S. Oa +3 @& 


~ 


_ 7 rt 


91URS OP SUIOS So] — BpeURD sjuRNeqUIOD sudTIOUY ral 
NUDAAI JJ INS JOdUIT,| 9p JUSTIA [| 2NUOD 9y)NT eT — EpeURD NUdAdy Il 
sjuenbul[ap sop uoneidepess op soumumessoid sa] — epeurd [eEUUONIaLIOD BdIAIS OI 


UONeIUSWI[S9I Jo UOTDadsuI : xnvUITUR Sap g1URS P] 
12 XNLIIS9A Sap UONSI}OId BT] — EpeULD oseJUOUITTROIZY 19 dIN]TNOLISY 6 


a[e.USO UONe.NSTUIUIpR | UeILIGe 


S[GNoUILUT | IP UONINASUOS kb] — 9}LINOgS 9p JUSWIOUSIASUdI NP USIPeULD ddIAIOS 8 
s[euoTyeU asuaTaq L 
BpeULLD [PUONUIOIUT IOIOWIWIOD Jo sogsuULID soMeITy 9 

dRJaI1g 


xied ev] ap uanuUreUl aT 


enbrqnd uonouoj ey] suep sajsod sap uonenyeag | op 19 UOTJVOTJISSLD LB] IP IUIOJ9I VT ¢ 
SOUISTULSIO SOT J9 S9IQISTUTU SO] SURP OUIOIUT UONBITJLIOA eT 4 

[eJopaj JUSWIOUIOANOS ne UOTeNTeAS 7] € 

[PISP2J JUSWSUIOANOS Ne J[eIUSWOUUOIAUD DOURINS BT Op aIANIO Ud OSTUI BT C 
UONPTJLQA ap SUOTIeAIASGO sonny I 


sjue]]Ies sjulog 19 sodoid—-juvay 


anideyy 


9661 IeI — | SWUIN|OA 


salaijew Sap ajqey 
966} — SCUNWIWOD Sap aiquieyd e| e epeued np jesouab ING}CIIJUIA 
np woddey 


9661 IP — epeued np [er19ues Ind\eoTJLI9A np woddey 


‘aygqnd np sajqiuodsip jJuos sjuaunozop 
xnap sa} ‘anbiuysaj uoypjasdsaqui, p 
awAof snos anb isuip nuadad a] Ans jodui,p 
a4aljoul ua aadioyuv uoisizap ap ausof 
snos aayqnd aja v uolnjasdsajui aja) 


‘DpDUD, ND JuppIsad 
alonpy vb] ap sajqosodun suaippuv9 


suaig sap juainja anbyqnd aja190s 

D] ap suoyav Sa] anb nj2Uuod Dv DpyudyD 
NUIAIY ‘SADUDULY Sap adajsiulu np 

an3ad uoluido ] ap Ja ask]DUuD UOS ap ajins 
D] Y ‘UoNYs189] D] ap NAdsa,] Y ausofuod 
yivja uoyvjasdsajul UOs anb saanssv,S 

ap uLfp S2DUDULY Sap adsajSUiU a] ajjnsuod 
D Ja aJXaJUOD ANA] Ja NUDAADI JJ INS JOCUWIT | 


ocr 


UONVOTJLIDA Op SUOTIBAIOSQO SONY 


9661 IRIN — epeued np [elpugs InoyeoIJLI9A np yoddey 


ap 107] B] ap sajuauysad suoyisodsip 

sa] ask]pup b ppouny nuadray ‘suousanb 
ap aduas a2 aapnosad flop [1,NbSAo] 
apnqvy UOS 18,9 aUOD “sansiquip 
juos inb 107J v] ap suoyisodsip sap 

ans asodas Ja axajduoo jsa uoysanb ajjao 
qwAguas Anavoifisad a] anbipul,] auuoy 


‘sajqvsodun suaippuns suaig sap 41uajap 
nad ppounz np juapisad un 1s 4auIusajap 
p4oqgn,p Inv{ [1 ‘uoisizap Vv] Supp sagsodosd 
suoyovsuns xno anbiyddp,s uoyisodsip 
ayaa 18 dautuajap ap ulfy ‘aadidd 9191908 
aun, p SUOIJID,p adUDYIA Ua Sanuajgo 

aja 1UO Sajja 18 Sajqpsodu suaippuvd suaig 
sap quos anbijqnd 3491908 aun, p Suolgv 
sap anb asizaid NUdAdI J INS JOdUIT | 

ap 10] Y] ap asaynoyavd uoyisodsip aur) 
‘saronpy sa] avd sanuajap aadtsd 9491908 
aun, p SUOIID,p asuDYIa Ua SalonpyL 

saj 4nd sasinbon aja quo anbiqnd aja1v0s 
D] ap SUOIJID Sa ‘JUuaWMOoU ad Y JVIddI 

ua suivs Sa] Ans Jodui,] vp Saijyjalnssp 

adJa SUDS JUDPISAd UOU adIDIIIf[aUag 

un v saionplf Sa] Avpd saanq1sjsip 

adja Juadnad sajja ‘pppuvy nv Juppisad 
saionpyf sa] nod sajqosodun suaipouvo 
suaig Sap JUOS SUOIJID Sad 1¢ “saionpif 

sa] 4od sanuajap anbiygnd ajaiz0s v] 

ap SUOIJID Sap aANJDU D] AUAIIUOT Sanpuad 
SUuoISi2ap Sap Jupuluusajap Julod aT 


‘aauuop Uuo1isodsip VD] aAnodj 

as janba] Supp aj]xajuod a] IssnD SIDUL 
‘asaynoyavd uoiisodsip aun supp asin 
21]2ql] 3] JUaWa]Nas UOU JUDLAPISUOD Ua 10] 
D] ap jiAdsa,] aulludsajap DpDUDD nuaray 
‘saaspsiaua suoyovsups] xnv anbyddp_s 
inb u0yp]s18 a] BD] JuDJaAdsaqul Uq 


‘UO1S12ap D] DAajIadsad DpOUDD 

nuaday ‘asiv{ a] ap asododd jivja j1,nb 

121 Saajajdwood aja JUO SUOIJIDSUDA] Sa] 

anb ja sjapxa Ja Sjajdwood quos ‘aadioyuv 
uOIS1Iap ap apubwap DV] SuDpP splAdap 
‘saasodoid suoyovsuvd $a] ja syipf{ say anb 
IULAYUOI UOIDIYIA9A VD] 1S ‘apudulap vb] ap 
jalqo,] juaipsin{ inb ‘saajaload suoyovsupay 
$3] JUIWAANAAIIN AALIAIA ap DPOUDD 


UONBITJLIZA Op SUOTBAIOSQO sonny 


ED 


nuaaay sod aysadwa,u jad ‘anpuad 
aja pv aadioijup uolsizap aun 1s away 


‘ansiquip siofapd 

jsa ayjanbv] ‘107 vo] ap uoyvoyddv,] 

D JUAWAALJDIAA ABYIUDA] NOP DpYUDD 
nuaday ‘aadiayuv uoisizap aun aspuad ap 
ULfY “SAUlJ189] SA]DIIAIUMOD SUOIJIDSUDAJ 
sap sanjoaff{a,p JUDAD apnjij4sao ap 

U10SAq UO Sa]QONg1AJUo) SaT ‘sajuvjsodu 
savas suoissnoiadad Sap Jaan saxajduoo 
SayDIIAAUMMUOD Sajuajua sap uauuanbadf 
quanbydun inb saajafoid suoyopsuv4y 

ap Juvjnozrap Sajpasyf sanuanbasuoo 

sap jalns nv sapnyijsaoui Sap D & 71,nbss0] 
saadiojuv SUOIS12ap Sap Juapuduap 
SA]QONGIAJUOD SAT ‘a8DSIAUA a]QDNq1AJUOI 
aj anb sasioasd suoypsado sanaisnjd 

ap NO aun, Pp PADSA,] Y NUIADI 9] INS 
}0dUII | op lo] Y] ap suoyisodsip sauinj4sa9d 
ap vat pinb uoypjasdsaqui,] asioaid 
asajsiulpy a] ajjanby] supp ‘ajqonqisquod 
UN VD ‘Aj1dIA UOIDADJIaP GUN JSa NUAAAA A] 
Ans Jodui,p asaijou ua aadioyup uoisizap 
aug) ¢ DpoUuDD Nuaday ap SanjuawmMuUo?y 


“SUOTSTIO9p Sa] NdoI JUO INb 

XNoo & QUUOp d3eIURAL | OP SoTqenqLNUOS 
sone .p gaud s10ae ynad sagstA SUOISTOSP 
sop ‘juowuaoer ve nbsnf ‘uoneorqnd-uou 
RB] ‘ulyuq ‘UoNdope a)109 op 

SoTJoMUsAg Seoudnbgsuod sa] Ins onb tsure 
aJa\sTUTPy a] Jed sayueysodumt suontsod ap 
uondope | Ins asAyeur ,p 39 UOTRJUSWINDOp 
op onbuew np juoulayese suojginbut 

snou SNON ‘s}OdUII Usd sIeT[Op op suOTT [TW 
sInoisnid MUDAP | B IDIANODII Op OIp ne 
QOUOUSL HR ,U SIIP--JS9 9 ‘9VOSTJ OOISSP | 
B ajuTONe gyI0d }1e .U epeuRD NUDADY 
‘suoneiado sao & d[qeIOAR} UOISTOOp 

oun juRpudl ud ‘anb ouOp suOUSTeID SNON 
“UONUDIUT 97199 9SNIF IOAR JUdANd [66] 
op uoIsIOep ve] ed sagsta suotesado soy 
‘SIAR aOU Y ‘sgsodurt ang JUSATOP suaTg 
$90 B JUDWIDATIPTAI SaTNuINSOR yeyIdeo us 
sures soy ‘epeued 9] yUoWINb « sayqesodut 
SUdIPeUD sUdIq » Sap onb sane sudtq sap 
anbsioy ‘to7T e] ayaAei onb inayeysts9] np 


9661 IRI — Bpeued np [ei9uas InayeorsLIA np yoddey 


UOTUDIUI | UOTIS ‘ssasestAud suoTyeiado 
Sap JUOJBTNOSEp Inb nusadi 9] Ins yodurt,p 
d1QNeVUI US sadUaNbgsuOd soy JURSsTULIeS 
Ud OpNjtIE9 uN sayqenqiyUoS xne 
yuouuOp sor[q ‘JOduT | sp uoNeNsturUpr | 
suep jueyoduit ayo un yuonol saadronue 
SUOISIDEP SI] ‘NUadad a] Ans Jodui,] ap 10T 
PR] op uoneorydde | yueuaoUOD sopnjginbut 
SOSNILIIS SP DAQTNOS UONBOTJLIGA 

ap uoNeAIasgo aJUaseid eT €9'1 


uolsnjouoy 


‘oniooe uoTWesT[Iqesuodsal oun Jed ormnpen 
9S JIRIAOP onsd0e doUdIedsuey oun ‘snjd 
oq ‘onbrydde ;s [eostj Wop a] WaWTUIOD 
dipusidusod v o1[qnd 9] Ja sorreyuawayIed 
SO] Ople I]JO steur ‘sIoT[Iasuod sng] 

OP 19 SITQENQIUOS sap dyde} PY OITIORT 
souaredsuey e] JUDWIOTNIS UON “[BOSTF 
dUIISSI Np sdudIedsuL] PI] JURSSTOIDOR UD 
No} ane | & B[qenqiNUOS uN ._p d]UdIEYoo 
snjd jo owuoj1un snyd toy ey op UoNvor{dde 
oun JouTenUd }IeIASp oI[qnd ne anTeUUOS 
oe} SY] ‘opeosTy UONeISTUTUIPR | 

suep jueyodumt aor un yuanof yodunt .p 
IIHeUI Ud sogdtonUe SUOTSIDEp So] 
“uoTUDJUI 91399 suoAndde snon ‘soedtonue 
SUOISIO9P SOT $9]NO} sastTTeuUOSIadap 
UOISIOA Ud JaI[qnd ap uoTNUSyUT UOS 
gououue & BpeUeD NUdADY ‘UOTIBAIOSqO 
ayuasgid ey ormnpold & yUeSTA [IeAR aOU 
ap sINod ne ‘GGG SIQUIDAOU UA Z9'T 


“ogsT[eUUOSIOdap UOISIOA Ud ‘966 | 

sIeu ud ‘agtTqnd og .p WOTA [66] Ud 
ONPUdI UOTSIDEp BT ‘SONpudI 939 WO sata 
NO JUSWIOU! Ne TdT SUO}eI] SNOU JOP SaT]ao 
Jorqnd sed ou g1gyoid & [I ste ‘suUOTSIO9p 
soureyao gtyqnd usr & dI9}STUTTAL 

aX] JOduII.p s1gNeUI US sonpual saedronue 
SUOISID9P Sd] ‘dasTTeUUOSIOdap UOTSIOA US 
‘arjgnd epeurd nusaoy onb spuewwosel 


SUOABR SNOU ‘C66 UA 19'T 


‘orqnd ourewrop 

Np 3S9 SQ6] US dQUUOp UOTUTdO | onb 
I9}]0U Op JUSTAUOD [I ‘aUOD Jeg “[66[ US 
onpudal uOIsIdep ey] snyd uou sed ang-jnod 


RUUD dU [I ‘so [I,.nb amnay_,| B “Ja JouUe 
SWIQUI BT ONpUdI sedionue UOISIOEp P| Op 
dourssteuuod no sed ong-jnoed vu epeurD 
nudady Jed gsoddo suas usa sguuop 
}OdUII .p sJgHeUT Ud UOTUTdO | CR6] Ud 
ndal v Inb sajqeiduroo ap oUuTged dT “S86 
ud ONpudl UOISIEp ek] sed jIessteuUOD 

du onbuooinb Ins souerAar .p Inoensuoy 

oun yreuUOp IN] RAD “soyqeiduos ap 
JoUTged UN Pp UONUDUT | & C6] Ud onpUdI 
1OdUI .p d1gHeU Ud sedIoNUe UOTISIOEp PI 
op o1doo oun WeAe [66] US onpusr ioduIT.p 
oJgHVU US sadionUe UOISIOEp eB] epueWOp 
B Inb syeooae .p JoUuIgRd JT 09°T 


SUOISIDIp Sa] NdeI JUO Ib 
XNdd & JUUOP 93ejUBAL | IP SIALId 939 
IIOA® JUDANA SafqeNnql4yjUOd sa.17ne .q 


“9101s oun ._p saAndde 

JUdTe19 ,.U JUSUOSeSUD | TU UOTWeIOUOUDI 
RIN °« Sotqesodun suarpeuro 

sudtq » sop sed yuoreja .u onbiyqnd gjar00s 
R] Op suoTjoe soy onb [eunqgi) un Jopensiod 
eunod epeuesd nusasy onb yos0 a1onpyy 
PT Is onb ngjea op &.U JUDUTASe3UA | 

onb s[quias snou [I ‘ano u_q “a[eJapey 
UONLIISTUILUPe | SOAR SOININ suOTLIOI 

Sos Inod oinjsod asTeAneul Ud 1eINOUT 

9] Jonbuew A Jeo ‘yuoWIaseSuUd UOs & sed 
yreronbueul ou syqenqiyuos gj onb esnf 

B IQSTUTJ] 9] ‘SJUDWUNIOP Soy JUI}S9}e | 
UWIUWUOZ “STUL)-SIeIq SO] 19 PpeuRD 

J] oUS oeOST] uonueAuos py zed onagid 
uoNeIgUOXd | WeNbOAUT 39 UOTITSOd vs 
yeasuevyo sfqenqiyUod gJ ‘ayIns ey Jed ‘Ts 
d10jNd9xXe sed jreIJas dU JUOWASeSUN UN,Nb 
JICUUOTUSU II}STUTLU-SNOS Np ddIAJas Op 
9}0U PJ ‘SUds OUIQUI oT SUR] “SdITO}NDExXa 
sed yUos ou o1U03 Np s}usWINDOp 

Soy oyfonbey] uoyeas ‘ayuouT1ed souspnidstinf 
RP] essteuuod [I nbsind ‘sayeosty saye001 
soy nod stg yea39}01d JUsWIASeSUA | onb 
JIBARS EpeURD NUdADY ‘snid 9q 6ST 


“UOISTOap 
eR] Npual & [I nbsJo] yey Vj JUTUVOIOJ 
jlessTeuuodel [I ‘« sayqesodut suotpeurs 


UONLOVLIDA Op SUOTVAIOSGO sanNy 


9661 IRIN — BpeuRd np [eIoues InsyeoLIga np yoddey 


suaig » sop sed juaTe}9 ,U suonoe sat anb 
‘UAB | SURP ‘UTOSaq Ne ITUaINOS v JgId 
1re]9 SIISTUT oT onbsing *« safqesodurt 
sudIpeURd SUdTq » Sop sed jUsTe}9 .U 
suorjor soy onb juvUa}NOS Ua UOT}eSHOS 
a][2Anou oun ,p gpuoj-uarg 9] Joanoid 
BIAOp EpeuLrD NUdAdY “UOTeIOUOUAI 

BI oyINs VI Jed Moyea oseJ nog ‘JoduI.p 
astyoury Ud STU) -sIeIg XNe sogIgysueI} 
ong ouop yuaAnad ja « so[qesodurt 
SUSIPLULD SUdIG » Sop JUSWISATIDATJO 

JUOS sUOTIOR S90 dTToNbe] UOTES 
uoneuyye | ganosdde v o1Q\STUT] 2] 
‘UOISIOEp PI JURPUdI Ud “IC *« so[qesodunt 
sudIpeUed sudIq » Sap sed UsTeI9 .U 
anbriqnd 9\9100s ey ap suonoe soy onb 
yuvUdINOS US JUSTUONbTUN seu ‘UOT}eST}OS 
J][9ANou ouN I]qQeI9,.p epeuRD NUSADY 

v yIeyouLIad aTja ,nbstnd ‘aurayser og}10d 
gun nb yeaev ,U UONeIDUOUL P| ‘STOJOINOL, 
‘Q[QeIOAR] UOISIOEp ev] op Indde J 

B OULINSSE OUTLLIIO OUN JIISTUTY Ne 
JUUOp JUO UOTJLIOUOUAI PT 39 JUOWIASeSUD | 
anb suossteuuodsel SNON SSI 
“OINUINOY 
ane oun MudAr | & JUaTeIpUsd syej Soy 
NO SBD SI] aUOD dINSOUT SUTe}IIN OUN sUep 
yreasgjo1d a] eyao onb 30 yUOUIO}TAQ-HUe 
a[eiguas a]8ar ve] Jonbiydde .p 1asestaua 
yrerinod jt sayjonbxne sainjny suonesado 
Sop JueUIONUOS opnjeInbur 93n0} WedIssip 
vjoo aonb gynole e araystuly a7] “JOdUIT | 

B OIeSNOS 9s ap oNA Ud suONRIadoO soy 
JonjoojJo .p sed yreasestaud .U o[qenqinuos 
3] onb yuaTeuLTyUOS UOTFeIOUOUAI 

RT 19 JUOUTASeSUA | “epeURD NUDADY Saiej 
® snou aonb suonesepoep soy UOTAS (ga | 


‘166 US onpuds JOduIT,p sIaeU 

Ud sadronue UOISID9p PT OADOEI nod 
dITeSS999U UONIPUOD oUN IsuTe JURSsT{duIoI 
‘UOTPIOUOUSI oUN Jo JUSWOSeSU UN TUINO] 
B JIGeNQINUOS 9] ‘9]UTe.ID 9)}90 JadIssip 

op ulfy “Od | B oTeNSNos as & JURSTA 
suoneiedo sop yuowayetoeds ‘apuewop 

e] Suep soguUONUSUI UCU suoTeIado 
sone .p anjoajjo .U a[QenqinuOS ay anb 


UONBITJLIQA Op SUOTIRAIASGO sonny 
SSS 


JIVUBTLIO dIQISTUTY 9] ‘“epeURD NUdADY op 
SIDISSOP So] JUSNUOW J] SUTWIOD 9ST 


9}UI9.1}SI1 ANI[VA 
JUN JUO UONBINUOUA. LB] 39 JUIUIISESUA /T 


‘JUSTeIA UD SoT]a,nb npouos 

JUO SUOTSIOSp Sap SasreYS soITeUUOTOUOJ 
So] sTeUI ‘« SaTqesodui suoIpeued suaTq » 
sap juarej9 onbrjqnd 9}9190s ey sp suonNoe 
Sd] IS JOUTULIOJ9p Inod suonor .p osueyo9 || 
gururexd sed yUO .U SUONROTJLIQA Sop 
SQSILYO SOMCUUOTOUOJ SIT ‘UONVOTJLIDA 
aun ¥ 1OINTd JUL}OUINOS So] UD S9gNDaTJo 
elap suoneiedo sap adno90 ;s d19}STUTJY 

3] “UOWATRULION *« sogjofoid suonesedo .p 
psese | & JUSWIATNIs sonpudI JUOIIS 
soadronue suorstoap soy » onb astogid 
epeurs NUdADY ‘7YO-OL UONeUIIOJUT.p 
OIIe[NOIID ey] sue ssl 


‘OI|STUTUU-SNOS Np UONUIIUT | 
B DOIAIJOS Op 910U PY Jed 39 SUOISIDEp Sop 
USUIeX9 Pp 91TWI09 aT daidope ke nb uoNtsod 
py Jed ‘juowrasesud | Jed ‘suorstogp sop 
s]Ud8e Sop SUOTILIOqI[ap Sop 39 SUOTUNEI 
Sop jua}stxo Inb snpuds saj}dutos so] 

ed si0J P] & QUITTJUOD }Sd LIAD “UOTSTIOSP PT 
ap Jonuassa | UdIG IeNISUOS 919}9BIED 99 
ap souasaid ey ‘suonoe sop « sayqesodunt 
SUSIPLULD SUdIG » Bp d19}0BIVS 9] INS 
jusulgssoidxe 9110d sed je ,u ‘anbruysey 
Q][oqQI] UOs yUoWONbruN ssaptsuod 

uO | IS ‘UOTSTOEp eT onb udTg ‘noT] no efap 
VIVAR SUOTIOR .p asULY OI | SIRI “UOISTOOp 
PR] DIpual op JURAR dIPNOsal B suONsanb 
soyedrourid sop oun | ey 11e19.d “oeaud 
d9190s ouN ,p suOTIOR ,p aIIeda.NUOD 

ua sostnboe 919 JUaTRAR SoT]o Nb 

gored « sayqesoduit suaipeues suarq » 

Sop juaTenyysuos anbryqnd 9191908 ve] 

ap suonor say onb uontsod ey] Ans yWesodas 
166] US ONpUsr UOISTOOp eT vs'T 


99nj}J0]J9 efap uoHessdo sun 
v sonbydde 939 & sod1aHUe UOISDap eT 


‘O1U9S 99 Ip SUOISIOSP 
Sd] Ins sa}duioo sap yusuonbryqnd sipuel 
ap UoTe3ITqGO UOs op JonINbow s MoOAnod 


9661 IRIN — Bpeued np [es9uad Ind}eoTyLI9A np Woddey 


sed ou op onbsti asa\sturyyy oy ‘syuouTIed 
sjoodse soy yuURIQUINUS JUDUNIOP 1N0} 

ap soudsqe | ua ‘anb suoAoI9 snoN “Nuadad 
a] {ns joduil,] ap 107J &| ap sasessed 
sonne p uoneordde | ins 0Ae .p wenbsu 
SIAR 199 op uONEIdad0R | aonb saduapIouUr 
sop onbuosjenb uonesrguinug oun 
JUSUTDTJUOS BpeULD NUDADY IP SJOISSOp 
soy] onb jeuou sANoN JUSWOTeSE SUOTINV 
SnON ‘uOoTIIsod op Jasueyo B epeued 
NUSADY QUOT B SOULUT,] Sop o1Q)STUTUT 
np ndai stav | onb y1ej 9] 19 nof us sowUWOS 
sop souryioduil | guUOp jUeI9 ‘UONeNTS 
31199 suep setynsnf juswaydure e319 9JUNN]e 
ONON “INSI9}X9 | op ndai stave J g1d990e 
UO BpeULD NUdADY op solTeUUOT}OUOJ 

So] soyjenbsa] snod suostes sop 

UONLNUDA SUTL}IIO DUN 3o SOOURUT Sop 
J19ISTUTU NP SdITVUUOTOUOJ Soy SOAR ONUD} 
uOTUNSI BT op Npuds 9}dWI09 UN ‘epeuRD 
NUDADY Op SIOISsop soy suep “JOANOI 

B SUOIPUd}}e SNoU snou ‘ono UY esl 


‘s1oduiI Ud sieT[Op 

IP SUOT][IW Ip SOUTEIUIO SOP ITUDAR J 

B JOIANODAI Op WOIp uN & JURSUOUSI 

Ud JOST] NOIsse | B OJUTANe Q1I0d 

ye ,U epeueD nudAdy onb suousIesO snou 
‘gnuoo Jeg ‘JaXed ve oduit un gurenue 
JUSUOIVIPSUIUMT JUSTVINV 19 S99NDIFJO 

d19 JUSTeINV sogsesIAUD SUOTIeIOdO SOT 
‘gnpudl aja sed year ,U d9pURUOp UOTSIOap 
ey Is ‘anb ajqeqoid nad js9 |] ‘owloug 
INITeA oUN,p SUdTG Sap ¥ SJNe[ar [eydes us 
SUIvS So] Ins UsIpeURd JOdWIT | op a}e,duI0D 
UOHVUIUNTTS | “MUSAv | Inod ‘Huvses 
ang-jned & epeuesd nusasy ‘« sarqesodur 
SUDIPPULS SUdIQ » Sop JUdTeI9 

suonor soy ayjonbery] uoras syqenqiyuo0S 

np uortsod ey yuejdaoor uq TST 


“xno JURADP 
agyiod snof un yso a}uUsuTVJed osneo oun 

Is ‘xneunqi) say sed onb aaysue.y ong ined 
au uonsonb eq *« soyqesodun suorpeurs 
susiq » sap sed yuorej9 .u suonoe soy aonb 
uontsod vy ins yueAndde ,s ua uoNest}09 
J][PANOU oUN I]QeIE WeAsp [I ‘uONeIOUOUAI 


PR] op ueyeagid as epeurd nusAdy Is 
‘Q[eUBIS SUOAR | SNOU dUTWOD ‘sn{d oq “10T 
ev] ap uoNvordde | inod suatq op a11089}e9 
ano0 & JusTeUdLIedde sagIapIsuod sUOT}OR 
SO] IS OIIP ap IITOLJIP 389 [I ‘« sojqesodult 
SUdIPLULS sUdIq » So] JUOUUOTUSUT 

mb nuadas a] Ans Jodui,] ap 10T &| 

ap sosessed sap airxo[duio9 vy nA, Ist 


‘« sajqesoduit 

SUdIPeURD SUdIq » sop JuaTe}9 onbi~qnd 
d9190s BI ap suoTjOR say onb sinj~ouOS 
Inod saoueUul,y Sop d19}STUTU Np STAR J INs 
gAndde jsa,s epeued nusacdy “guuonuoul 
SUOAB | SNOU SUIWIOD *« sayqesoduil 
sudIpeUkd sUdTq » ap sortejotIdoid 

ang juassind sjuaptses-UOU 39 SJUSpIses 
aonb yreAOAgid Indye[STS9g] 9] ‘91N9] 

91199 Op UOIsNJOUOD kB] UOTIS “sUOTSN[OUOS 
S90 ap SaTJaMUaAP SaTeosT} saoUsNbesUOd 
Sd] INs oN9] BS SUBP SagdUOUY 
suorIsnpouos soy otndde mb uoneyuournsop 
QUNONe IDANOI & ISSN9I ITOAR U 

QIL[OEP B SNOU 19-INJa_D ‘ssoueUT Sop 
aIsTUTU Np JURUDAOIA [G6] VIQUIDDEP €Z 
Np s9}ep o1)}9] UN Ins sgpuoy 

QUIQUI-9]]9 11819 IO] BP] Op JouUOp Jed 

TUly & epeueD NuUsdAsdy onb uoNe}9Id19}UT | 
roAndde ve yueAsos osdyeur 7] OST 


‘uoIstoap ey JoyNsnf nod dnoo sgide 
d9SIpal ajo & osATeUL OUP) “sUOTUNSI xne 
yuasgid sed 11e19 ,U SUOISIO9p Sop gsreYyO 

epeued NUdADY op dTeUUONOUO}J InNeY 

a] onb gnbieulas suoae snoy ‘Jueyoddes 
AS suoIstogid sane .p IU “aIquigdep €Z 

3] edronsed yuo sosreuuoToUOj sas 

op suIe}199 o]jonbey| ev uoTuNgI eT op Npudl 
ajduroo op snjd uou sed yusuusTU09 ou 
SIOISSOp Sas onb gjeusts & snouU ssoURUT 
sop s1o}stuTU of ‘Jed es Inog “jueyIoddes 
AS uorstogid one ouNone Tu suoTuNeI 

$90 op npudl a}dur09 UNONe JUdUUOTUOS OU 
SIDISSOP SOs onb SPULIOJUT & SNOU JQISTUTY 
YX] ‘QLIOALJ UOISID9p ouN gSTIOjNe UO 
epeurs NUdADY Op sosTeUUOT}OUOJ syney 
SOP ‘T1661 PIQuIZ.=Ep ¢€Z 2] Sonud} suOTUNEI 
ap dLas oun suep ‘sToJaynoL, 6r'l 


UONLOTJLIDA Op SUOTJBAIJOSGO saNY 


ee 


9661 IRIN — PpeuRd np [ei9Uas InsyeoILI9A np Woddey 


‘anbyqnd aja190s aun, p 
SUONID SAS ANAJIAJOA PLINPIA 
D angiAjJy ajoywuvdy alonpiy 


ayer y prey 
aronply 


ea? aure1olygueq ap HOI Y 


anbiyqnd g}at90s 
(anbiyqnd 9)9190s Y oun ,.p suonoy 
aun ,p suonoe 
sop Juouusidui095 
sudiq so7J) 


991.1999}01g INP 


¢ ade} - ajqenqiyuo9 IVIOTIJQUIG IP NOI 
aj Jed saanjoayja suonesedo 


aarernyguag 


PS"| 899ld 


SIU) -SPIQ 
XnD 4a]JVISUL, § JUOA ADIJIAIOT (anbifqnd 9191008 
JIINP1 A Ja adWIIIfaUag IT oun .p suonoe 
sop jusuueidwi09 

SusIq Sa’T) 


ayeruey aronply 


VAWNVO aUTeIIYQUDQ Op OIG 


991.1999}01g 
afponply 


AUIBIDIJQUIQ OP 1101 


Zz ade}3 — ajqenquyu0s 
a| Jed saenjoaya suonesedo 

aalepyguag 
9G"| 89ald 


UONPOTYLIZA Op SUOBAIOSGO sonny 
eeeeeeSSSSSSSSSSSSSSSSSSSSSsees 


9661 IRIN — Bpeurd np [esaues InayeorsLIOA np Woddey 07-1 


(anbyqnd gigts0s ‘OOIAJIAJOAT 
sues eds a1onply supp asinioifauaq ap 

sl at a pata jloap un, p ayspdasjuos ua ‘jodun,p 
Se sey) ISIYIUDAL UA ADIAJIAJOA AIINPIY 
agyermue yg aonpry D adafsSUuDA] 1SA a]DYWUYT AlINply 
SUuDp asIvIDYfauag ap JlOAP aT 


dUIBIDIJQUIQ ap NOI 


9911999}01g 
epnply 
‘T661 
ua aajuasasd nNuadad a] Ans Jodui,p 
dILIOIJQUIq Op OIG adaljDUl Ua aadiayuv UOIs1Iap ap 
apupuap DI JuvaAn alyoupsf advjiq 
ade PIU 


| edey3 — ajqenqijuos 
a] Jed saanjoayja suoljesedC 


qS"} 899d 


(anbrjqnd 9}9190s 
aun .p suonoe 
sop juouusidui09 
sudiq So’q) 


ayeyrmey aronpiy 


onleIoljgugq op 101 


alley guag 


aleijiul anyonsyg — ajqenqisyuod 
aj sed saanjoayja suolesedC 


BG"| 899ld 


UONOILIDA Op SUONRAIASQO SONY 


Aiat 9661 IRIN — epeueg np [eigugs MojyeoyLIga np yoddey 


‘onbipuinf star | LOSo1 UCD Jatauel ¢] 
C661 


‘JUSWINSeSUN | 


}2 UONLIOUOUAI PJ W10SaI [I Ja 9pULLAP UOISIOEp LP] PUdI epeURD NUdADY IIQWIIIIP PZ 
“Q[QUIOARJ 
UOISIOap aUN dIPUAI & 19 UONRIOUOUAI oUN jo JUDTIASeBUA UN Ja}da00R & 
JUASUOD dIE\STUTJAY a] nb sgwuIOJUT JUOS dTqenqlijUOS Np sjuLyudsgIdal so] IIQUIIIOP €7 


‘Q1QVIOAR} UOISIO9p dUN IIPUdI ap je} 

v [I,nb x1oyo Np spuoj-UdIq NP doURINSse oUTe}IZO OUN BpeURD NUdADY 

B JOUUOP B Yas 91}9] PT *« sayqesodult suatpeuvs sudlg » op sameygtidoid 
ag JUdANad s}UapIsal Soy “INdJLISIS9] NP UOTUDIUT,] UOTas ‘anb yn~ouoCS 
SIOULUL SOP IIQ}STUTU 9] *91}}9] VT SUR “« SaTqesoduT sudTpeURd sUdTQq » 
$9] JUDUIBDUOD suOTSOdsIp sa_ ‘oleyye | Inod aye1onso soueyodur oun .p 
suoT}Isodstp souteysao Jajdopr v Inoje[sT3g] a] assnod ev nb uoNUs\UT, | 
juesodxa o1}2] UN sdoURUL Sop o1Q}STUTU Np }10SaI epeUeD NUdADy JIQUIIIIP €7 


“e[RISp 2} 

UONeSIUTWUpL | S9AV SOINJNJ SUOTJLIAI Sas Inod ainysod asteaneur 

Ud }eIWOW 3] Jonbueur A Jed ‘JuouIeseSua UOs kv sed vionbueU ou 
afqenqiyuoo 3] onb asnf epeue_ nusacy ‘JUswWaseSud [9} UN,p o110INdaxa 
21Q}OVIVS NV 39 QYIPT[LA ev] & JUeNb spnjHIDOUI OUTL}IZO BUN Pp 9dUdISIXI | 
‘somerorpnt sosnes sap Jed ‘assteuuoo epeued nuardy onb ualg ‘vag 
‘Q[QRIOARJ UOISIOSP OUN IIPUdI ap Jd UOTeIOUOUAI oUN Jo JUSUIESeSUS 

un Jo}daoov ,.p asd UOISIOgp Bl ap SULIOJUT }S9 SUOISTOSp Sop UsSe J IIQUIIdIIP €7 


(‘UOTUNEI PB] JURUIZOUOD suOTsIOgid Tu soleyUawg|ddns sjusueUSIasuar 
Tu yeanoy Au [I,nb ja sJoIssop sas guriumexe rear [I,nb npuodar 

v epeur) NUsAdy “UOTUNYI PI Ins suOIsIdgid sop JoUUOpP snouU ap epeULD 
NUSADY &B YPuULLUSP SUOAL SNON “JOISSOp ne sINSIy ou UOTUNEI Lv] 9p NpuUdaI 
9}dW09 UNNY) SIO} dUN sJOOUD JUASsTUNEI as SaTIeUUOTOUO] syney soq IIQWIIIIP ¢€Z 


(uotung1 

P| JURUIEDUOD suOTsIogid Tu somejUoWg[ddns syuotWoUsIasuUa TU JUSIRANON) 
AU S[I,nb Jo siolssop sing] guTUexe JuaTeAR s{I,nb npuode: Uo sazg\sTUTU 
xnap So’T ‘UOTUNaI v] INs suOTsIogid sap JoUUOp snou ap ssouRPUL] 

Sop S1Q}STUTU NV Ja LPLURD NUSASY & 9pULUTOP SUOAR SNON “JOISsOp 

he sinsij ou UOTUNSL | ap Npuar aJduI0d uNoNY) ‘ssouRUT sop aza}sTUTL 
NP SOITPUUOTSUOJ Sop SdAK JUASsTUNYI as SaTTeUUOTOUOJ siney seq IIQUIIIIP €7 


(‘suorungs sa] JURUIADUOD suOIsIoaid Tu sarteyuoUIg|ddns syuouIoUSTasuar 

fu ANON Au [I,nb ja sJaIssop sas suTUUexd WeAr [I,nb npuode1 
B epeUe,) NUSADY “SUOTUNDI Sa] Ins suotstogid sap JoUUOp snou ap vepeuRD 
| NUSASY & gPUuLWAP SUOAR SNON “JoIssop ne sIN3y ou suotuNesI sap 
(ayINS) "| edaIg npusl a}du09 unony) “stoj xnap yuasstunal as SOITUUOTIOUOJ sjney saq IIQUIId9P €7 


—_—_— 


UONLOTYLIDA op SUOTRAIASgO sanNY 
cies 


9661 IP — epeuRd np [esl9ugs Ind}eoTJLIVA np yoddey 


8-1 


‘gfqenqiyuos np uontsodoid ev] aonb 

SUOT|N[OS Sane Pp IdUOUS JUOWINIOp aT “epeuLD Np Inati9}xa,| e sonbiyqnd 
S9J9IDOS OP SUOTIOR ,P WoJsuL np jeFJ Np a[qvAed prespustAsp Jodut un nb 
SIAR Pp JUR}9 DIQISIUTPAY 2] ‘aTquidaooe sed jsa.U spqenqiyUOS np uoTIsodoid 
ep onb onbrpul a]]9 39 SaSTAQI SA SIJSTUTLU-SNOS NP IOIAJOs ap 9]0U BT 


‘Q9STADI OQ Op o1ysTUTWI—sNnos 
3] Inod saBIpeai VdIAJAs op ajJou v] onb JUEprIoap sorreuUOTJOUO}J syney soq 


‘onSal jso onbipunf sav .p syoneges cup, 


(‘stoJ osgtuaid ev] nod apuvursp ve] op sUIOJUT 9}9 B aINSTUTLU-snos 

3] ep aJenb e yrenbrput.u Jatssop 9] suep ust onb npuodas v snou 

UO ‘UOISIOSp ap spuULUAp LI Op 9ULIOJUT STO} o1QTWIAId v] INOd 9}9 & 10-INT2V0 
puenb ja osturu-snog ne agAOAud 9}9 & 9}0U ve] puenb epeuRD nusAdy 

& QPpULUIAP SUOAR SNON) “JdISSOP 90 sULP I[QLVIOALJ UOISIO9p oun spur 
sed jnod ou argystuly a] aonb aNsturu-snog 9s] suUIOJUT epeuRD NUSADyY op 
jurofpe amsturut-snos 9] a[[enbey suep o1AJas ap 9J0U oun p JuatUasstquIg 


‘uaIpeuRd JOdUIT,|  JUDUDIQIUD SITVISNOS as op a[qenqmUOD nev jreoULIEd 
nb ‘stup]-syeyq Sa] 39 VPpeURD 2] 9IJUD o[VOSTJ UOTUSAUOD eT Jonboaut sed 
aU ap JUAIJJo sjuryuasardal sar] *[ 66] e1QUIA0EP O| NP aAFJo may 10}do99" 
ynod ou [I,nb afqenqijuos np sjuvyuasaido1 xnv OARS He} epeURD NUdADdy 


‘sue burs JueInp suonoe sap Jasodsip sed ou ev JuoWesesue 
UN }9 UONLIOUOUSI OUN ITUINOJ Op JWUdIJJO S[] ‘ONPUSI BIOS OU a[QRIOAR] 
UOIsIOap ouNone Nb sauLIOJUT JUOS s[qenqIUOS Np sjueyuaseidal soy 


‘1O'T BP] SWIQUI-IN] Jo}91d1OJUT JOSsIe] oJ] Op opuvutap IN] [1 39 [QVIOAR] UOISIOEpP 
ap sed vipual ou [I nb soouvuty sop oJa}sTuru 9] WWJoAR epeueD NUdADY 


"Q[QVIOALJ UOISIOSp ouN aipual sed 
ynej ou [I,nb oprloap epeurd NUsAdy op SUOISIOSp Sap UdIUILXa,p 9}TOD aT 


‘QIAINS B& OJINPUOD Op BUST] L] Ins UOT}epURWTTODAI SUNONe s[NOdEp Ud,U [] 
‘gIquIad.Ep € NP 21}}9] ve] Jed sagaaNos suonsonb sap Jaynostp-inod juasstunel 
dS SIOULUL Sop S1Q}STUTW Np jo eEpeuRD NUdADY 9p SareUUOT}OUOJ soq 


(Z)Spz oydeisered 

3] Jonbrydde ina ap JuepueWUOdaI US suOTIeI9dO $99 INS JUSTOTAg-HUR 
J[vIgUuesd d[3I Ll Op a}TWOD Np UOTUI}e | JoINNVe Pp “NJ JJ [I,S 39 
IOPULUIAP I[QUVIOARJ UOISIOEp L] JOSNjol op UONUSJUT UOS JUUONUSU Inq [I 39 
SOISRSTAUD SUOTILIOdO SO] SOOULUL Sop dIQJSTUTU NV O[BUSIS epeuLD NUDADY 


‘T'] eogid ey e omsy 
Inb ‘cg6] Ue onpudl ‘nUdAdI J] INS JOdWII,p aIgVeUI US dadIoUeR UOISIOSp 
PR] ap atdoo oun epeuryd nusAdy & WUINOJ B]QenqiyUOD np juLyUesgIdal aT 


UOISIOSp op epuRrWap s]][aAnoU Pv] ap UONda.0y 


“salIpowW spuvulop oun sed 

agov[dural ‘1 66] eIquiadou / 9] ‘sind saItjol Jsa opuLWap 9}}9_d ‘oITeIOIyouaq 
uN ,p OUT LT & Naty] OAR agyndai ‘uontsodsip vy onb sure ‘sue [7 

soide spuvyoreul najea oysnf ey & nat] OAR agyndai ‘uortsodsip ev] Io}Ag 

B JUASIA Sd9SeSTAUS SUOTIeIadO Sa] “UOISIDap op epueUep v] op UONdadey 


IAQUIIIIP €Z 


aAquadap OZ 


1QUI999P 6T 


dAQUI99P ST 


AGUII99P BT 


JAQUIII9P OT 


1quiad9p Z] 


2AGuiad9P ZT 


JIQUIIIEP 9 


JIQuIadEpP ¢ 


JAIQUIIAOU SZ 


JAQUIDAOU / 


SIVUI Q 


1661 


anpua 10s au 

L66} 8p nuaAas aj uns 
yodwil.p auaijew ua aedionue 
uolsidep ej anb juene sadeje 
sajediouiid sap aibojouoiy9 


b'} 899d 


UONVOTLIGA Op SUOTFAIASQO SONY 


Pa Tad | 


966[ IRIN — epeueD np [ei9ugs Ina}voTJLIA np yoddey 


aS 0) 

NP pie39 J B I[QRIOAR] UOISIOSP Op sIpUsT 
sed y1eAop ou asa\sTUTAy oy onb njouoS 
sosudar sinatsnjd e jU0 sarreuUONoUOJ 

Sop ‘apnig,| & 919 B UOISIO9p op spueUrap 
ev] sjonbsay] JuvInp stoul xnap sap sinoo ne 
‘anb JUd[2AdI BPeUeD NUDADY op SIOIssop 
So] SUBP SNUd}UOD s]UDUINDOp sa] grt 


jodi, p soumwi0s 

$9}UL}IOMUM,p IUBAB | & 191ANOIII 

dp WOIp ne 9du0UAL 9.199-jned &B 

epeuey NUdADY ‘UOISIDEp P| JURpUIA UY 


“« Soyqesoduit 

SUDIPLULD SUIIG » dP ISSIBL;S 

ou [I,nb surour eB ‘epeued 2] Jud}INb mb 
SudI XNP JUOWOAT POI sa~NuINoR [e1Tdeo 
Ud SUIS So] JosoduIt,p 1so Inb ‘mayeyst3eq 
np sfeigues UONUIUI | JUSNSNIy 
suoresodo sao aonb suokoio snou ‘snd 

aq ‘sked np ojos ima] B suonoe sao ap 
uOT}Isodur | sure JUeUdAgId Ud 19 EpeuRD 
Np JUapisss un Jed sonuajap onbiyqnd 
919100s aUN Pp SUOTOR Soy « satqesodurt 
SUSIPEULS SUSI » SdP dLIOZ9IvO vy] 

SUEP JUL}OUUpe Ud INo}e[SIS9] Np UOTUAIUT 
91199 9TSNIJ OAR JUdANed [66| Op 
UOISTOgp B] Jed sagsiA suotesodo say ‘stAR 
aou Y “s]UapIsgs-uou sap e JusWonbiun 
« sa[qesoduil susipeurs sustq » op 
souruoyedde | asestaud Ioyeystsg] a] onb 
eNUOW 10] e] op [esougs uryd 9] : oJUeAIns 
uontsod ey] aAtsensiod uodsey op ayuosaid 
(ZT a0gId) Cg] ua arumoy uorurdo | 

anb gnbseular suoar sno Lr'l 


‘usIpeuro 

19duit | & o1IeNsNos as ap syuaptsas-uoU 
So] Joyoeduis ev JUDTO}sayIUeUT 

aSTA — (I(T) Sg BouTTe | — sagatid sagi00s 
op suonor .p ansedanuos us sandal 

JUOS Saj[o NbsIoy « sayqesodur susrpeuvo 
SudIq » Sop JuaNINsuOd sonbijqnd sa}9190s 
ep suonor soy a]fenbr] ap nyJ9A us 107] 

e] 9p uontsodsip vy ‘anno ug *suontsodsip 
S99 ap oted Joy [EZ] apOTIe. 19 (L)STT 


UONRILIZA ap sUONRAIASgO sony 


se 


19 (¢)LOT soydeisesed saq ‘syuaptsei-uou 
sop & JusWIoNnbiun « satqesodurt suatpeurs 
sudiq » op sdueusyedde | asestaus 
INIJLISISY] I] ‘TOT KP] Op spqurasud | 

suep ‘onb Jasuad ap suostes souuog 

op JusUUOp IO] ke] op suontsodsip sanne 
sinatsn{d ‘sdurs} surgu uq *« safqesodunt 
SUSIPeULS sUdTq » op omejatIdoid ang 
nod epeued np juaprises un,nb a1t019 ap 
hal] suuOp B9UTye | ‘TOT [9p ZO] aonse | 
ap sIQIUIN] PT B 19 JUOWATeIONT] g}9Id19) UT 
‘stojoynoy, ‘aieyo ned s_quias ang,p 
UOSILI BS STRUT ‘TOT RP] SURBp sonuUdUOD 
JUSUWIZ[NOI ap suontsodsip sanne ,p 

v puodsaii09 ‘souuarpeues souuosiad 

dP S9}9190S Say Ins 910d tnb ‘egurye 

JOD ‘nuadad a] Ans JQduN,] ap 10T e] op 
(9(Z)L6 BeUT]e [Ns 9] Ins jesodal sooueuUry 
Sop SIQISIUTU NP NSdI SIAR J Or] 


‘« sotqesodunt 

SUSIPBURD SUdIQ » Sop ITUa}9p JUdANed 
sUapIsar sot onb uortsod snod a\dope 
epeue, NUDADY ‘SsoouRUT Sop o19)STUTUT 
Np ndes stav un Ins yueAndde ;s ‘a\dur09 
np jnoqg ny ‘juaprsas un Ied sondar 

yuos sonbijqnd sajgt00s ap suonor soy 
anbsio] yuowaye3e onbrydde,s a8essed 90 
IS JOUTMLA}9p OUOp WeAOp epeueD nusasy 
“suonjoe sa] Isure y105a1 Inb yUaptses-uoU 
uN js9_.9 onbsJoy JUWIO\soFTUeUT 
onbydde .s sagaud sgjato0s ap suonoe .p 
atjiedanuos ua sondoi yuOs saTfa .nbs10] 

« SoTqesodulr susIpeuRd suatq » sap 
Juanj}suod sanbiyqnd sajgtoos ap suonoe 
sa] Janbnp nya ua (I(T )¢g BeuUTTe. [) 
nNUuadad a] ANS JQdul,] ap 107 ek] ap asessed 
3] IO “epeueD ne yuoreptsgs soronpry 
XNop So ‘aduRYyo9 | op JUoWIOUT NY ‘agALId 
919190s oUN .p sUOT}OR .p atIedaNUOD 

ud onbrfqnd gy9190s aun ,p suonor say naz 
HEAR gfelTUey aIONPLy [166] IP UOIsIogp 
PB] JUBAR SaguUe sonbjand Sr’ 


‘« so[qesodul suotpeuro 

sudtq » sap sed yuos ou sonbitqnd 
S919I90S ap sUOTIOR Soy ‘JUdTUATRIDUaH 

"« satqesoduit susrpeueo suaiq » sojadde 


9661 IRIW — Bpeueg np [esgues InayeorytI9A np oddey 


ibe | 


Nv JOpIsal op vsosseo aI] ,nbsso] sperm oronpry suep uonedionsed es ap gsodsip 
Tloav deindgi ‘(1 )gp eydessesed np uonroydde med ‘sed esas au soLNda}01g S1IONplT © 


‘QOLOOOIg VIONply & operpiue{ o1onpr{ red 
ser] onbiqng ep suonor sap uonnqmme | e sed eronbiyjdde_s au (¢)/0]| oydesisered a] @ 


(Q(Z)LOT BOUTTL | B JUSWIWIOJUOD QUILLIO}9P INOS UN B SUOTIOR SaUIQUI So] 
sinboe 110Av agyndai vias 901LN99}01g atonprj onb jo (e(Z) LOT BUTTE | B JUSTUQUTIOJUODS 
agUTULIA}9p SUTIUOS OUN v g[e39 oNIedaNUOD oUN INod ‘ssanqine sey] onbiqng 

ap suonoe sop gsodsip soar ogyndail vies oyerrue, aronpry anb suowyuos snou 


suonor sap (nol na MoAv egyndgi uoNtsodstp 9yn0} sLidwtod A) yos 99 anb uoltsodstp 

anbjenb g JueueAneyal nuadad a] Ans jodu.] ap 107 | op (P(TSTI 39 GCDOIT seoure 

Sep NIOA Ud SpuRUOp ouNone viojuasoid aU o][2 yUanbgsuos Jed onb 30 ‘uoNUsATOD 

PB] OP TTX apouse | op p oydessesed 9] ronboaut sed ou & ‘p-k-"o ‘soanquyje 

ser] enbrjqng ep suonor s9j Ins sgpnumnsov yeyde us sules sep B JUAWOANLIAI 

(« UONUSAUOD » I) S1UQ-SIDIT Sa] Ja DpoUDD a] asjua CYS] ap ajvIsif UOUaAUOD &| 

Jed onagid uoneiguoxe aunone Jonboautu e ‘stuA—sIeIq Sop JUepIsel uN JUdTAQp ato 
NO dep LI & JURSUOUTIIOD STOW (Z| ep epoted oun Inod ‘98eBua,s soLde}01g a1onpljt e@ 


‘soguue XIp Seo op SINOD ne sgjnuue 

}IOS dU UONeIOUOUAI 93390 aonb sues ‘oguUOKUSW UONISOdurt .p sguuR | JUBAINS soguUR 

XIP So] JURSIA [SUOISIO9P Sep UOTE P| & SMUINOJ JUStMETVS9 vIos UONVIOUOUAI 

ay00 ep aidoo oun] (1)(e(p)ZS] B9UT[e-SNOS Ne JUSLUIQUIIOJUOS opT[eA UOHeIoUOUAT 

aun ‘so109}01g S1ONpLy & 997] enbiyqng ep suonoe se] enquye 1o-a][99 e[fenbr] 

ap sinoo nv g[elMue,y o1onpLy ep uonisodump eguur | & (¢)LO] eydeasesed np 
uonevordde | mod ‘epeued nusacy op JOLIsIp ap nevoing ne juINoJ ojerjruey stonpl{ 


: Sar[duraI JUSIOS SaJUBAINS SUOTIPUOD soy onb nAInog 


“epeura 


: SOJULAINS SUOTSIOEP So] SUOpUeI snou 


‘soonqie ey] onbijqng ep 


‘SUdIG Op SOIUDS SUTe}IOO B[SOI 91199 

B UOTJda0x9 JUO,] “SUITG S99 B JUSWISATILIAI 
IL6I stndap sgynumnoor jeydes us sures 
Sd] steel oak seyndgs ‘ayesigues s[59I 
ud ‘So JosuUeIIO | B SUDTQ Sop oIgJsUR} 

nb ja epeued ne oprses Inb atonpry 3yn0} 
‘QUUOTJUIUI SUOAR | SNOU SUTUIOZ prt 


IN9}PISISI] NP UONUsZUT | 
J41j}snaj ALOAB JUdANd suOIe.19d0 SaT 


S9aA9|NOS SUOI}SENH 


‘g[qenqLUOS 9j Jed sagnjoajyjo suonesgdo 
so] ajuasgid aja ‘¢'] a0erd eT B URN? 
‘UOISTOSp 91190 sIpuas Inod soryouesy 
sadeja sayedioutd sop o1sojouoryo 

gun onjnsuos f° | a0eId ey] “¢'] a0eId 

PR] B OINSIJ [66] Ue onpuss Jodut .p 
d1geUI US dadrIonue UOISIOSp eB'T er'l 


‘T661 Ue 
sked a] ayinb yuo suonoe sao onbsioy 


epeues ne joduit | e smofnsse ang up 
duop JUSTeINe sgTNUINdOR [eIded US sUTeS 
sat onb ja « sayqesoduil susrpeues suaiq » 
sop sed quarey9 .u onbryqnd 9391908 Pe] 

ap suonoe sa] onb y1eJ oT Ins yURpUO] as Ua 
UOTVSNOS ][aANou uN IT[qeI9 39 UOTISOd 
BS JUDUTOIgTdUIOD JOSULYO JIeIASp [I 
‘UOTIBIOUOUAL LI Ip We[BAgd 9s o1Q\STUTJA 
3] Is ‘onb Juonbrpur uoNnerouousl vy] je 
BpeUeD NUdADY 9p SJOISSOp So'T ‘UOTIeST}OO 
QI[PANOU JUN IIa 9UOD IN[qeI9 .p 

epeuerd nusacdy ve Joutod mb ‘sue xIp 
inod apryea uoNeIoUuOUdI oun TUINOJ 

& operrue,y oronpr. “ed es inog crt 


‘epeues np 
yedgp uos op aiep PJ & saynuINooR [ede 


‘aaaldd 9Ja190S 

aun, p suoyav,p aijspdasuod 
ua Sandad aja uo Sajja,nb 
aosind « sajqpsodun suaippuvo 
suaig » sap yuos anbygnd 
2191908 D1 ap Suoyov sa] anb 
jsa aquaonl-snos asayjodky ] 
S861 2p uolsiogp B] SUDP 
aUUloy ‘a2uoua Jao aanosddp 
spd JIDAD_U ]1,8 UO1S19ap 

ayjao npuad Spd JIDAND,U 1 
sipul ‘« Sajqosodu suaippuvo 
suaig » sap juos anbygqnd 
219190S D] ap SUONID Sa] 

anb spd auuoluau au pppuvd 
nuaray ‘Uoisi2ap Db] SuUDG 


L661 8p nuedad 9] ANS yodwi,p 
asalyew Ua sedioijue UoISIDEp 
B] ap anbiuysa} 21}9q!7 


"| 899d 


UONLOIJLIGA Op SUOTBAIBSGO saNY 


LS 


SI-l 


966] Ie — epeurD np [esouss Ina}edIJLI9A np woddey 


“snssop-19 

a]LID9p uoSey eT op sosinbor ve say sronpyy vy enb sored Juswe]durs juepisei uoU arterolyauaq 
Np safqesoduil susIpeuRd SUdIG SEP SUTUIOD SdgJOpIsUOd sed jUIeIJOS OU SNOA Ied saguUOnUa 
suonoe soy onb StAv,p JUSUIS[VS9 OUOP SOUIWOS SNOU ‘Sa9pI,P 2IPIO SUIQU 9] SURG “(Q}UsUTIIOd 
g[VOSTJ UONUSAUOD P| ap JULTNOOYP UONLIUOXE 9]NO}] Bp 9AJOS9I SNOS) UsTpeUeS JoduIt | 

ap suIy xnv sast]iqeiduroo og JUSWs[VULIOU JUSATOp “JUsUIaTeUTy asodstp Ue UO NbsIoy ‘Inb 
suaig sap (agindai JususfNas No a][99I) UONIsOdsIp e] Ans oJed eB] Op no uTeS np UONLIe}sSUOD YY] 
JaIgffIp ap youtad “1oT eB] ap (1) gp oydessesed 9] ouru0d yo} ‘107 Be] ap (S)O] eydessered oJ 


‘(S861 [HAR 9Z 9] JUBAR soNINSUOS UONLIOdIOD ouUN JUOUTaTqQeqo:d) 

epvue, ne sed yuepissi ou UoNeIodI09 9uN Pp sed 9] Jso ZOUUOTJUOW snoA onb s1UES 2] 

sIvUl ‘SUONeNIS op Satuas xnop JUatWa}sajiueU aSeSIAUD A INOIeISISQ] oJ ‘epeuLD np Jwepiser 
un & a[qesodun uorlpeuro uslq un.p souvusyedde | a8estaus ‘oT ve] ap (O([) EE] voulye,| 

RB ‘INajeISISg] o] Jonbe] uojas JusUINSIe 910A suOANoIddesap snou ‘juR}9 LJEe_D “sa[quiquies 
S9TJOq!] Sop JUsUUSTUOS I0'T B| op (11)(T P(T)8OT eeutye-snos 9] 39 (¢)L01 cudesSered 

oT (« epeuryD nv gpisas ‘sauur | op JUSWIOUW UNONe & ‘JIRAR_U a[QeNqLUOD 9I Is afqesodut 
UoTpeueo UaIq UN jIeIEs INb Ustq UN » sANOI UO] NO ‘IO7T LT op (e(]) gp voUTTL,] B “xo “d) 
SJUopIsgI-uoU sep B satqusodull suarpeues susiq op souvuayiedde ,| o8estaua InaqeIsIS9] 2] 
anb aquour 10] ve] ap [eigues uryd 9] ‘zoyeusis 9] snoA auTWIOD “p10doR ,p sed saumuIOs ou sNON 


nuaai aj Ans jodwiI.p auaryew 
Ud S86} Ue eauUOp UOIUIdO,7 


Z| adald 


np j0 (1) 1°$g eydvaisesed np njy0A ua o1onpy ey op sarqesodun suorpeuvo 

SudIq Sep JUeNysUOd saqfa ‘« enbiqnd uoneiodios » aun .p suonor-eyideo np a1039}v9 
oUN Pp Sestug suoT}oN sep QOT ‘d ¢Z surow ne jUs}UesgidoI saguUOUoUE suOTOR soy] Is ‘aonb 
aipusjus eB ZouUOp snoA ‘Wenbrddes to7T ek] ap [Sg ajonze | [anbne o8uvyoe un Ied sasinbox ev 
a][9, nb suonoe sap yuepIses OU aIIBIOLJgUEQ UN Y oNgLye oJULpIsaI STONpY oun No seo oJ suep 
« 9]qesodun usrpeuvo uolg » uorssaidxe,| ap UONRIIJTUBIS I INS SIAR aOU ZopuRTAp sNOA, 


TO] BL ap (x1\(Q(T)S TT eoutfe-snos 


Ud SUIVS Xnv JUIWIANPIAI UoIpvuLd NUdADI 
9] Ins jodui1 | v yUOWTSIgHUS oITeNSNos 

a8 op singe Jed yenjoutted my mb 
STU(]-S}¥1q SOT 19 BpeURD I] aNUD oTROST] 
UONUdAUOD 2] ap asne[o eT Jonboaut sed ou 
B “SIU()-S}81q Sap s]USpPIsaI onuUdAap JIeIOS 
aq]9 aqjonbey B a}ep PL] JURAINS sue (| SOT 
SUBP SUOTIO® Soy epUdA JIA Is ‘aase3ua 
1S9 S$ (QOINIIJOIg s1ONpI4) STUP-sIeIq xne 
JoT[eISUT,S preyye mb aronpy ey] Wl 


‘(¢°] a0eId eT JIOA) UONRIOUOUAI oUN 

}9 JUQUIIS BSUS UN ISsTUINOJ d[qenqLNUOD 
9] onb uontpuos ey v ‘agdionue uorstsep 
PT NpUSl & oIQ\STUTPY 9] “saSesTAUD SIOTL 
uonesado | e jUaJoYUr (g¢"] oydesSered 
ne gnboag) onbsti ay yuouroyreyed 
Wessteuuos epeuRD nUdAdy ‘UOISIOap 

B] OIpPUd op owgUI JURAY Or'l 


‘o9Atid 
919100s oUN _p suONde .p onedanuod 


UONRITLIZA ap SUONRAIOSGO sanNY 


se 


ua anbiyqnd 9}9100s eI ap suonoe soy ndor 
Weare sonpyy e] o[fonbey sed ommatiajue 
uonessdo oun sins yekndde ,s uonewayye 
aneaD *« so[qesodull suatpeues suatq » 

Sop JUSTeNIT}SUOD (QUUaIpeUeS sIONpT 

oun) s[eI[TWe.y s1onpry red sonuajap 
anbijqnd 9391908 oun ,p suonor say ajonbey 
UOTES J|quNqLynUOS Np UOTeULITTe | 
gidaooe & epeuRD nudADY ‘aTquIOAr] 
UOISID9p ouN JURpUdI UG 6o°1 


"SUTeS Sd YST[RAI IIeINe 

991}99}01g atoOnply onbs.soy ‘stup-syergq 
xnev sogigjsuey suonoe xne syneya. 
sgjnumnooe [eyideo us sures say Jasoduit .p 
WOIP UNON JUSWIT[ONVIA Nd Irene ,U 
epeury 9| ‘STup)-s7eIq Soy 19 ePpeuRD 

J] OUD OTROST] uonudAUos PR] Op asneyjo 
9UN Pp NAA Ua ‘anb sIyTUsIs eTa_ ‘sue xIp 
9p sutow stndap y1e}stxo epeuRD of WeyInb 
tb atonpry eT ‘stozayno} ‘seo 90 sueq 
‘anbiyqnd gj9100s aun ,p suonoe sourey199 


661 IRIN — Bpeura Np [erlauas MajeolyIA np Woddey 


View 


TOT P| Op (TUY(Q(T)S TT Pourfe-snos ay Jed jo (1(1)¢g vouTTe | Jed sgsta sopqesodunt 

SUSIPLULD SUdIQ Sap JUOIONINSUOS suoT}Oe sao onb sored ‘vag “yuoreusjredde my mb 

aaaud uoNeIodio9 oun,p suotoe ,.p ansedanuos ua sasinbor v ajja,nb senbijqnd suonesodi09 
ap suonor sas ap gsodsip s0av agyndai ‘to e] op ({) gp oydeisesed np uoneorydde 

ted ‘sed vias ou ‘epeurd nv Japisel op viosseo a]9 nbs10] ‘atonpyy vy] aonb so UOIsIogp ANON 


‘(UOISIOAp RB] SUP « dTRITTUIe,] OIONpIy » 
agjodde) epeuesd ne juepisas stonpyy oun .p 
‘MOAQOAI WRAP SIONPT 99999 ‘STUL)-S1eIq 
Sop dUAPISaI oNUdADP 9.9 side nog 
‘(UOISIOEP BT SUP « 9d1I}N9}01g SIONpI » 
dgjodde jsa atonprj 91399) stup)-syeyq sop 
QUNpPISSI QUUDTAOP Jo EpeURD ne JOpIsol 
ap assoo o1onpry oun nb ye} np ‘NUdAAI OT 
Ins Jodu ,.p o1aneul Ud ‘saouanbasuod xne 
weyodder as [66] SP UOISIS9peT —- SET 


S8OL oP UOTSIogp P| 

Jed d9sTA UoTenIIs P| WeADTNOS onb saT[ao 
JUSTIN SA9AdTNOsS suoT}sonb sayedrourd 

Sd] STRUT ‘SJUdIOTJIp JUdTe}9 sIdgid sjoadse 
SUILIIOZ “SQ6] US ONpUdI JOdUIT,.p sJaTeU 
ua sedronue UoIstogp eB] op 9191j9UEq 

yuvde sarysied xne dgi[ai s1onpry ane oun _p 
ajduroo 9j Inod yrepuewop ey mb syeooaer .p 
JOUTgeS UN,p UONUDIUT | B UOTSIO9p 

91109 NpUdI & epeueD NUdADY *JOduII,p 
ISTYIUBAT UI IIQHUOAT BI AISAIABTY 
quareanod sanbiyqnd s939190s ap 

suo1joe sop onb A9uyu0d nod onpua.t 
99 B JOdUIL, p a1IeUI Us vadIonUR 
UOIS|Pgp eJeanou sun “T66] UA Le'l 


‘gouinsaid opnjyoeXoUl 93}99 Op SUITOJUT 
aj sed sinofno} year ,u sguUOp 939 

ev uorurdo | nb e sayqeiduros op youtqes 

9] ‘UONROIJLIQA orOU ap Uy eT Y “anbruyoo) 
uvyd 9] Ins aj}oRxoUT sUTUTOD UOTUTdO 33)99 
JULUDJUTLU dIQpIsuod [I nb gyeusIs ev snou 
epeurs nusady “‘UoTUIdo 93399 B JOALIIV 
ud Inod TAINS JUSWIOUUOSTeI Np gpuoj-UdIq 
3] JonuUoU assind mnb Jatssop 9s] suep 

uot & A,U [fT ‘ourgut UoTUTdo | 39 UOTUTdO .-p 
apuewiop ke] JUoWTaNnbrun JyUONUOS 

JaIssop oj onb npuodai ev snou UG_ “sguUOp 
uorturdo | otndde ,s ajjanbey] ins asAjeuer | 
JOUTWURXS NINOA SUOAR SNON OCT 


TT] a0aid ee omnsyy 
uorurdo /J ‘suonetodio09 say sed yuestA ou 
SUOISIO9P SOP UOTSIAIG, PB] Jed sarusnoj 99 

XNOP $d}NO} JUO UOTSTOEp PJ 30 UOTUIdO /T 
‘epeuey np jUepIsel un ,p « sotqesoduit 
SUSIPPULD SUDIG » SOP IUIWIOD SagIapIsUod 
ang sed yuoreanod ou sonbryqnd 
suoleiodios ap suonoe soy aonb sourysqns 
Ud JIeSIP a]]JO : OTUINOJ 939 & UOTUIdO ,| 
“SQ6] eu UG ‘sofqeiedWiod JUdIe}9 SjIey 
So] JUOp seo uN.p jofns ne (epeuRD NUdADZY 
sed yuo ou « suoturdo » soy ‘« suOTSId9p » 
xne yUsWOIIe.NUOD) UOTUIdO .p 

apueulsp oun ‘sajqeiduloo op youtqeo 

ane uN _p ‘ndal v epeuRD NUDADY ‘C6T 
ap sedronur uolstoep ek] op UONRoTUNUIWOD 
PR] Soide ouIvUlas OULE *epeURD 9J 3U9}3INb 
sate nbs.10] yoduit,| & satjefnsse 

yuos sonbryqnd suoye.10d.109 ap suonse 
Sa] ‘9[QU[qUIAS SBD UN B JUDJUIIANL[AI 
‘SQ6] U2 IUINOJ NUIADA JJ Ins yOduIT,p 
319 Ud UOTUIdO 9UN HOLS sel 


‘T] o0gid ey e ommsyy off ¢¢g6] Jorauel 

ua sanbrunurw0d 919 & UOISTOep 

eT] “« safqesoduit suatpeues suorgq » 

SOP SUIUIOD SddIEPISUOD JUSTeIJOS saT]o Nb 
gored joduil .p astyouesy us epeurd 

3] Jontnb yuoreanod o1onpry ey Jed sonusjop 
onbrjqnd uonesodsoo ey] ap suonoe 

soy onb yuewyUuOS ‘sgpueWap UOTSTOEp 

eR] Npusl & epeueD NUdADY Pel 


‘gouapIsal op sXed op yrerosueyo 

atonpry ey onbsio] sastyeas one soyndas 

sed ouop JUaTeJos OU SUOTIOR S99 INS [e}Ide9 
ud sures say onb ja « sayqesoduit susrpeurs 
SUDIG » SAP UIWUOD SddTAPISUOD JUOTeIOS 
atonpy oun red sonuajap sonbiyqnd 


S861 ep nusda, aj ANS yodwi.p 
auaijyew Ua vedioijue UOISIDep 
2] ap anbyuyde} 91/9q!7 


bb 899ld 


UONBVOTJLIDA Op SUOTFRAIJOSQO SONY 


9661 IRIN — BpeuRD np [eIsugs InayeoIyLga np yoddey 


suoneiods0o ap suomoe sauteyos 

ayjonbry uojas ‘epeurd nusaoy op ued 

B] Op ‘UONeILOYP 9UN Usd }1e}STSUOD oNTNOA 
UOISIOEP PI Op [aNUasse J “JUOT[D UN,p 
ajduioo gj inod pg6] Us UOISTOgp 3909 
gpueulop eae sopqeidwios ap yourqes 

UQ ‘joduII,p sstysuBAy Us BpeUeD) 

a] 4ay9nb yusteanod (« suones0d.109 » 
sagjedde s1oje yUate}9 $9}9190S 

say) sanbyqnd suoyes0d.109 ap suonse 
sap ‘Sg6] Us snpual JoduIT,p s1Q1eUL 
ud sodioue UOISIDap 9UN UO[IS Eel 


‘epeues ne soyel[ruey soronpy 

sop suep snudjep sIe[[Op op sper] 
XNop SUIOU Nv ULTRA SUI ap STU/)-sIeIq 
XN¥ SLJOJSUB] Sap B JUSWIOATLIOI 
‘sorqenqiuod sop InoAR} US SoNnpusl 

919 Juo Inb ‘nudAdl 9] INs JOdUIT,p sIQHeUT 
ud sogdronue suorstoap soyueyoduit 

XNop SUTUIeX9 SUOAR SNON (43 | 


"xTOYO [9] UN IIey 

ap soronply xne WpsazUI JUUEssaIdxo }s9 
[I ‘o]UeSIeJsHesS gJoINS oUN [eUONLU NUDADY 
Np oSTUIU Ne ITUINOJ op UOTIPUOD 

RL & « Sofqesodul susrpeued suslq » 

Sop jusuUstAep sonbiyqnd sgja190s op 
suorjoe sap onb Mstoyo ‘toy eB op njJ9A UO 
‘yuoanad sXed axou JuaInb tnb jo epeued 
ne jUapIsal Inb s919190S say 39 SJoTNonIed 
SY] *« Sofqesoduir suarpeurs susiq » So] 
Tued sagssejo sed juowayeigues JUOS ou 
sonbijqnd s9j9190s ap suotde say ‘aU0d 
Jeg ‘sojqesoduit suatpeurs susiq soy rusted 
SO9SSETO JUSUTATVS9 JOS SddALId Sg}9TD0S 
ap suoToe sayq ‘oydwioxe JUaT[IOx9 UN yUOS 
Ud BPRULD NP SANIIS SIOT]IQOUTUT sUdTq 
SY] ‘SUdIG Sdd B 9IT9I JOduAT No} B WOIp 
UOS IIO][VA dIIVJ OP dINSOUI Ud BIOS EpeUeD 
3] onb d]]9} 1sd 91103919 91199 ap sUdIq sop 
oINjyeU VI ‘aTeI9UaS OS] aq *« sarqesodult 
SUSTPRULD SUdIG » Sop onJed JUOJ ‘aTONpTy 
oun e juouUoTedde Inb ‘ares ureys00 
UN Pp SUIT SI S[Nas ‘stoJaynoy, Ic 


‘QIQHUS SINSWIOp SUUDIPeULd o[ROST] 
aHorsse | aonb uarq ts ‘939}01d 1sa susTq 
S90 INS BpeURD Np OIp oJ ‘Wp yuoWANNY 


UONVITJLIDA Op SUOTIRAIASGO samNY 


SS 


‘JUOpIseJ-uoU UN Jed snudjep sIOTe JUOS 
SUDI SO] IS OWIQUI ‘sasTTRdI JUDWISATIDATJO 
JUOIOS sUTeS sad onbsIO] (Ia3UeT9 

sked UN d9A¥ ONTOUOD s[BdST} UOTUSAUOS 
aun .p naa us Jonbrydde ,s yuaanod 

Inb suUOT}BIQUOXS SOP IAIOSEI SNOS) SUSI 
sao Ins yeyideo us sures soy Josoduit .p yop 
UOS DAIBSUOD BpeULD 2] nb jeyJ og] Jed 
anbrdxa ,s uordaoxe 93192 *« sorqesoduit 
suaIpeues suatq » sgjodde suarq 

ap dLI089}eO oUN ssa 91199 B UoTdooxo 
Juo,J ‘“sousnbgsuod us sgsodut juos 

19 S9STTRAI 919 OAR sgyndgs JOS soIgJsueN 
ISUIY SUDIG XNv JUSWIANLII [16] 

stndap sgynumndor [eIded us sures Soy 

anb js9 a[ei9ugs o[3aI PT ‘JosueNS | B SUDTq 
sop juorgysuey inb jo epeued ne jUuSpIsel 
Inb soronply Sop sed oy sueg iy sat 


‘TeyIdeo ud sUIes Soy 

Ins joduit,| @ o1TeNsNOs as ap afqrssoduit 
ang WeIAop [I,nb yuoWsTe WeuUUOdII So[SeI 
sapueis soo op sij{duuts a[sue | snos 
[Boslf OUITZ9I aOU asesTAUS oNnbuOSIN?) 
‘JoSuLIE,| & DOUSPISEI BS II[qGeI9 

Inod epeues sy sytnb suuosiad oun -nbsioy 
SQSOCUIT OUOP JUOS SI 39 S9ST[BII 939 ITOAR 
sgindgs JUOS sNUd}9p SUSTG SUTe}IOO suBp 
sgnunooe sures say ‘snjd aq ‘souenbesuos 
Ud SPSOdUIT JUOS SI 39 SasTTeaI 919 

IIOAR Sgyndgs yUWIETeIQUAS JUOS S9d9P 

UOS op JUSTO Ne JUSTIEp [I,nb sustq soy 
SUEP SQTNUINIOe SUTeS So] ‘Opgdep epeULD 
np jusprses un nbsio7T ‘gpnwnooe ures 

9] Isure asteas [I,nb yo (augte | “Pp-e-"d) 
udlq np « asodsip » yUaptsal a] onbsJO] 
gsodurt jso ules oy ‘epeued np JuspIses 
un, p JURATA nq ‘jOdUWIT,| Be smMofnsse jJUOS 
epeuesd np sjuaprtsel soy Jed snusjap susiq 
Sop INI[eA op UONeJUSWISNe oUN Pp IUIOJ 
PR] SNOS [6] Seide say~numnsoe [eyIdeo 

Ud sUTeS Soy aonb 4s9 ‘nuadas a] Ans Jodui,] 
ap 10T &| suep mny ,pinofne ajsojrueul 

as aT]9 nb ayJO} “InaVe[sIs9y_ Np UONUIIUT | 
‘sgid suondaoxe soureyias Y én | 


“sudTq 
oP S9I] SNUdADI Op oJeTTNOTIed swWIOJ ouN 


966] IRIN — epeued np [esguas noyeotyLI9A np yoddey 


cl-T 


‘TeyIdeo ua sured say oddesy yodurt [| aonb 38a 
omne up ‘aouaprsal op sked np uoTouoy 
uo onbrjdde_.s yoduir,| anb ysa sadrourd sao 
op uA “xneyuoUrepuoy sodriourd sureyis9 
ims gsodar [I steur ‘anbrdwiod jso NUdAAL 9] 
ins joduit,p Usrpeurs ouIs9I OT 871 


[eyides ua sures sap uotsodull TJ 


‘1oy ey tonbrydde .p 

JUOUIOATSNTOXS JUSTASI IN] [I “INSLI9}X9 | 
ap suoruido sop spuewop epeued nusAcdgy 
onb uslg “S9gStA }IOIp ap sjurod sap suuop 
19-Injao aonb onbrpunf uoneigids9qut | 
anreuuods Inod sonsn¢ ke] Op 19}STUTW 9] 
juoWIETeS9 aI[NsUOD |] ‘OduIT | JURUIBOUOD 
}IOIP ap sjutod sjussajJIp JUSpUd}-snos 
nb geosty onbrytjod ap sedrourid say ins 
SIAR SOP S9OURUL Sop oJQ}STUTUT Ne stTojred 
opurulap epeurd nusAdy ‘esojdope jt .nb 


uontsod ey JourMajap op ulyy ‘JoduIt | 

® SJNRISI WOIp np 3a anbytyod ey ap ay1edxo 
ashyeue oun o}Issoogu sedronue UOISTSap 
op SopuvUap Sop JUSTE) aT L771 


‘ureyioout ayquias onbrydde .s 

Inb yeosty JUSWIITe) OJ NO suOTFeNITS 

sop Suep sdgpueUap JUOWO[eI9US JUOS 
Sol[q ‘e8estauo [t.nb sastogid suoneigdo .p 
saTeosty ssouonbgsuos xne juenb 

apminses oun sfqenqinuos ne jusUUOp 
sogdronue suoIsiogp sa_ ‘soTqenqLuo0s 
sop ued Py op uoNest090jNV | suRp 

19 IO] PT op UOHeor{dde | suep gyuOjIUN | 
mloAnowioid op onb tsure nuadas 

a] Ans Jodui,] ap 10T ke] op d11teUOTOA 
UONPAIOSGO | JOM [IOVJ op UTJe NUDAVI 9] Ins 
jOduUIT,.p oJQT}eVU Ud sogdrIonUR sUOISIOap 
Sop pudl epeuRD NUDADY 97° 


91X9]U09 


‘suoisigap Sa] nda Juo nb xna2 p guuop asDJUvAD, | 

ap sajqonqi4juos sasinv,p aalid 410AD Jnad Saasia SUOISIIap Sap ‘TUauMuaIed INO} 

vp nbsnl ‘uoypvoygnd-uou v] ‘a4jno uy ‘suomsod $a2 ap sajjanjuada sanuanbasuoa 

saq ans anb isuiw ‘asajsiuyy aj 40d sayupjsodun suoyisod ap uoudopn,] 4ns asdjDUuD,p 
ja uoypjuaunzop ap anbupu np juawuajpsa suojginbul snou snoN ‘sjodui ua 

S4D]JOP ap SuoI]MU Sanaisnid AMaAd,] Q 42AANOIAA ap NlOAP ND qupsuouad Ua apDISif 
aljaissp,] 0 ajuiayyo ajiod j10.u ‘suoisizap Sa] Juppuad ua ‘Dpouvy) nuaday anb duop 
‘suou8ip49 snon ‘Tondvs ua sums sap uoiisodun,) ausaquod inb a2 ua Anajyjsi59] Np 
uoyUalul,] a4jsnAf JuO suoIsi2ap Sao 4nd Sagsia suolDosgdo Sa] ‘S1AD a4jou Y ‘Dpvun?) 
np sajoynun{ saionpyf sap supp snuajap juaija inb ssvjjop ap Spavyyiu xnap sujou. 
nv JuD]DA sualg ap s1uQ-sjory XxNv jAafsuD.1 a] JUDULAIUOI SagdiouuUD suoIs1I9p 

xnap PUNUDXA SUOAD SNON ‘soaSpsiaua suoypnsado sap yuosajnozap inb nuadad 

aj Ans joduil,p asain ua saquanbasuod Sa] JUassyUvévs 19-SA]]9) ‘saadioyuv 
suolsi2ap Sap DpoUDD nuaday Y sapunwap juadnad spi ‘ummjsaout 18a 19dul | 

ap sulf xnp Juawapivs] a] Juop suoynsgdo Sap JuasDSiAua Sa]qunqisjuod Sap anbs10T[ 


epeues ne sopeluey SoNply sap suep snusjap 
syjoe .p SABT[OP 9p SpAeIT][U xnap sulOU Ne .p 19d8Ue.1}9,] & JAIJSUBA} NE sonbiydde aya 
B nuadad a] Ans Jodull,] ap 10T B| yop UoSey eT B yUENb epnyginbur,p syyoul xnalgg 


uly K1apog : ajqvsuodsas AnajooyfisaA 
oly prypys : Juiolpy jpsguas anayvoifisaA 


epeuey nusrsy 


UONLOTLIJA Op SUOTJBAIASGO SONY 


EEE IEEE 


List 


9661 IRIN — Bpeued np [elouas InoJeolUeA np woddey 


‘Vdd VD] 2aAD asiAad UO1INGIAJUOI 

IP PAOIID,] Y SAGABIIUI JUOAAS 19-S$9]]9I 

10 WOAdV.] ap Juawmaoupuy a] nod 
SA]GISSIMIPD SINOI ap alalJDUL UA SAD1AJIAAIP 
S9U81] XND SUOI]DAON AUD SAdJnD,p 

919089U ASSOIT-a]JaANON VI ap nvaing 

IT ‘1a4aul, p sqyfuoo ap adaljDU UA SaJIIAJS 
SadJIadlp Sausy sap aidopv v YWddO 

D] ap [laSuOd aT “SIDIINSAA Sap A1uajqgo,p 
anovdpd vs saadofuad nod Wqd) VY] IaAv 
I]JIDADA] ASSOIT-A]]AANON UA WOAdV.1 

ap nvaing a] Ja aula] B8UO] DB SA1AYIV Pp 
uvjd un asndadd VD WqdQ VT ‘Sajuvjsodu 
juaumaddojaaap ap sayyigissod 

sajjaanou sanaisnid ap sajiford p ayosayo 
Ja SAAIDIVIO] XNDAANOU ap Y Sjuauiyg ap 
alQuiou UIDJAAI UN ano] VD ‘uo1wyjI0]dxa,p 
SJNOD Sa] adinpad Anod J1Dg np saygnd 
SIDIALAS SA] BANIINAJSAA D ja <aUulADUL 

D] ap SJapvo sap UO1YULAOL ap alqUua 

un sajiojdxa Anod sapjjop ap Suoijjiu 

xnap ap januup Jp4Juod un nuajgo BD Wqd) 
D1 ‘Dlad ‘SadWSSaIaU JUOS SUOIJDAOI AUD 
SadJND Pp ‘JDAQUAS ANAIDILIsIA 

3] AUSIINOS a] aWWUOD ‘SIDUL ‘Sa]qDsapPISUOI 


sa1sodd Sap 10{ D Vdd) VT 


‘ajaidosd ap JloAp Np JAa{SUDA] 

9] JUDAD asDg DV] ap Suaig ap Juawmaagpua, ] 
ap jalns nv saajnuof sapnjainbu 

Sa] Ans ajgnbua asivyyMu ad1Og V1 oéguas 
ANADIYIAIA A] BUBIJNOS a] AUNUOD “S1OU 
SAaUsap SAP SANOI ND sajivf suoyvsa]o 
Sap atquou spd jipsiA au ajja ‘juanbasuoo 
Avd ‘Ja ¥ddO PY] ap uoyvotfisaa aun, p spd 
NDSS18D_.S au I] ‘Yap a2 adajas D (WddQ) 
UONDIIOSSY Juaudojadagq YAV_ SYJVMULOD 
DT 090] Juauaddojadap ap adnoss un 
4Anod ajj1p] ap Yap un jsa suakopa say anod 
Sajqvanp siojdiwa xnvaanou ap saad2 ap 


UONVITJLIGA Op SUONRAIJASGO sonny 
—— SS? 


uyfp sakojduia 69¢ a10jdwa inb Ja savjjop 
ap SuOl]IU Q¢ ap snjd Juamatjanuuv 
asuadap inb asipjyjiu aspg apuvss aun 
AQWAO{SUDA],  BIUABY,] AP SAAIDJUIMIMOD 


“UOTBIOOSSY , | DOA 

npouoos p1osov | Inod JosgiL, np [lesuoDd 
NP SOATOOIIP So] JUSWISIENUA TAINS sed 
BU ]]J9 19 UONRIOOSSY ,| 99AR UOT]NGLNUOS 
dP plod9v | SUBP SUOTIIPUOD soyuR}IOdUIT ,p 
DINJOUT,p STUIO B BDUdBY [J ST 


"SOSIOA PIVAR OIJO,Nb spuoj sap sIe[[op op 
suoly]iu /‘z gsuadeap yna asJatusap 97399 aonb 
saide nb uonerdossy ,[ op SUOTUD}UT Soy 
JUBALIONP S9AT]IOV .p uvTd op ndai vu OTI[O 
‘snjd oq ‘uoneloossy | Jed uowsoueUly 
np uornesiynn | & jueNb sameyjo soyuaye ,p 
gnutoy sed vu s0ud8 YT P71 


uoIsn|au09 


‘orejuowiayddns 

yUITUOOURUTy Ud ¢ YOO OST esinogep sindap 
® O[q ‘SOOURAR _P ISTUIAI LT 19 SopuRUap 
Sop JUITD}IE.N 9] Ins someyuourstddns 
SoTQ.NUOD op dovy{d ua ost PT sgide 
‘QATLIIUL 9}}90 op JUoUIOUPUT] o] stdor 

B o0Ud8V [966] JorAUelUq = ETT 


‘g0ud8 VY ,|[ Jed stuINo; spuoj 
sop asuadap ke] suep $}919}UT Pp S}I]JUOS 
Sd] INS sddLNOAIp sousiy sop sanoidde 

B UOTVIOOSSY ,[ Op UOTeSTUTWUIpR ,p 
[l9SUOd J] ‘QG6H] SIE QZ Np UOTUNEI oUN 
B nb souojur B snou s0Ua8Y /J "6 GN | 


‘Q66[ 1QuId.Ep Ud QUTULIO} 

ong WesAop urd aD ‘suonryyeisul say nod 
DWI9} SUOT B INd}SeIIp ueTd uN JaIOgRIA .p 
MISSOI9U PT [QuI9 SaANoR _p ueyd 


9661 IRIN — Bpeued np [elsuas InajyvorsLIgA np woddey 


3] ‘ajduiaxe .p ann vy ‘someyuousyddns 
uonroyrueyd oun ,p Jo sapnyd.p 

a100u9 JUapusdap uryd np sajo sjoadse 

Sap ‘sIOJaNo], ‘doudsV ,| 9p SUOTIPUOD xne 
IOWLIOJUOD }S9_S UONPRIDOSSY J IT 


‘gyeuoT}eU dsUdTOq 
RP] Op SILT[OP op suUOT][IW g Op UOTUDAQNS 
PR] ap Jo UoNeWOTdxa | op jueUdAOoId 
$9}]9901 sop UONesTTNN | Inod sw419} 

Suoy snyd ev sargioueuly suorstagid sap e 


‘jouuosied np 
yeyor .p sanbryeid say ja sayjo ,.p sjadde soy 
‘tojdwid | Ins ‘surowl 9] No} B ‘JURIIOd 390 
UONeISIUTUIPR _p [Iasuod 9] Jed saganoidde 
‘satqeidaooe sojliog JUNWIOUUONOUOJ 

ap SopoyjoUI op UOTFeIOGRII,[ 


SUOT]VIOOSSY .| Op UONeINSTUTUIpR _p [TaSUOS 
9] 19 JouUOsod a] aQUS aTqeidad0R soyoR} 
Sop UOISIAIP oUN ,p JUSUTASST[qQuId |. © 


‘atqeidasor 
SoTATOR .p ueyd un,p uonesedaid eye 


: $agjdodsaI JUSTOS SdJUBAINS SgId 
suOnIpUod say onb 30 ev nbsnf yuowsoueuly 
Np 19Ue | Op “COG1 C1quiaydes TT] 

J] ‘UONPIOOSSY | SSTAR & OT[O ‘UOUIeXO 

100 Op dULID} NY ‘sJe][Op op suo] p‘Z 
ap soiejuswstddns spuoj op opuewop 
oun nda OAR sgide UONeIDOSSY , | 

op JUSWINUUONIUO}; d] QUTUTeXO 

B aouasV | C66] Oe .P Ul PTY 0c T 


“UONPIOOSSY , 
ap UOTIVIIO eT] SgIde STOW pT IOS ‘CGHI 
DIQUIDIEP jo] 9] ‘SPANO .p uvyd un nda 

® a2 nbsioy aonb spuoj sas ap sgsodoid 
uOHeSTTNN | INs seqoys s139}eIs ouN NdoI 
BU d0Ud8Y ,| onb je} Np suojginbur snou 
SNON ‘d0uUd8Y ,] op NSaI WAR .U aTTO Nb 
snjd op uol]iu ¢‘(Q Os ‘sa[qIssTupe sjnoo 
ud SIe][OP op suoT]]Tuw Z7‘¢ INO} Ud adeSUO 
WAR oI[O,Nb reyeusis uoNRIOOsSsSyY | ‘S661 
JIQUIDAOU (IE NP aJep UY “UOTT[TW 9O*T 

ap souvar | & soyndul ve sOUNSY ,| 

anb ‘sjapeo inod dures ne yey sed 
juoTeAr ,U Inb ‘say[qisstuupe siesy sone .p 
asesuo & UOTILIOOSSY J ‘o[eUOTeU ssUuZTIq 


Py red goueuly JUDWIOIONUS 939 O}INS 

py red v inb ‘sjapeo inod 939,p dures un 
Jotoydxoa assind aya nb urge uUoNRIdossy | 
B QOURAR 9}9 B SILT[OP Op UOTT[TUT 9°] 
‘JUBJUOU 99 aq ‘a0UdSY | op aanoidde 
yUsWBOURUTy Np QOT “d 9¢ No sie][Op op 
suOTT[Mu /*7 N5oI WeAR UOHRIOOsSsyY | “C661 
NOR C] I] “SIOW sIoy op suorstAgid sop 
INs 1a Sagsesua sasuddap So] Ins sapuoj 
919 JUO UONeIDOssY | & SJUdUaTed Sa] 
‘gipuodas A yo onbruiouod9 yuowraddoyaaap 
OP dIONeLU UD XNBIOT SUTOSAQ Sd] JOUIIS 
Inod on[noa spnyyey ey We UONeIDOSSY , | 
anb 910s ap NSUOd 939 RAR UOTIVIOOSSY , | 
d0Ae ploooR .| onb snbrpur yuo snou 
d0udsV | op sjuRjUasgidal saq 61l'1 


‘aseq DUUDTOUR | dp SUSTQ SUTe}II9 ap 
UONPIOLINIEp PT JOYDEduId _p 39 OULID} SUOT 
B JUSUIDUUOT}OUOJ Op $]NOd Sd] JOTO.NUOS op 
y1e19 sosuadap sad op jofqo J onb sgurojur 
UO Snou s0ud8Y ,| op sJuRJUssaidar 

So] ‘SUOTIeSTTIGOWUT xne suONeIOI[SWIe 
Sap &B 30 ‘SUOTIRT[LISUT Sop UdNI.NUA | 

B 19 UONeTOTdxa | B “UOTeNSTUTWpR | 

B gIoRsuoD JUOWETedrourLId 

D9 B OOUNSY | op JUdWIOOURLUTT 

a] aonb g7e]sUOd SUOAR SNOU ‘sTOJoINO], 
‘QUIIO} NOD ¥ sasTTNNes 3.9 JUSTeANOd 

Inb synoe sop uonestfyn J saAndde imod 
SOITRSS9O9U SINOS xne JUOWIIOUPUTJ UOS 
JULITUNT] p10d0e | SUBP ISNeID oUN SNOUT B 
J0udsV J “aseq P| ap syoe sop UoNeNUS .p 
}9 UOT|S98 Op sIeIj Soy] Nod UOWIOOURUTI 
ap MOAgoaI sed presnod ou UONRIDOSSY | 
anb gstogid & Jose], np [lasuo_d 

J] ‘souesy | Jed spiosoe JUsWIOOURUTy 

3] ganoidde e [1 ,nbsioT SII 


“sogoueUuly 
ang JUIIeASp UOTVLIDOSSY | op sosuodap 
soyjonb Jsuruiajap op 10-a]]90 Inod 
QIOVJIP Sot) 919 OuOp kv |] ‘eoudsV | Jed 
JUSUIDOURUTY Ne SoTqIssTupe sioT~Nonsed 
S]NOD Sa] sed yessIUljap su [Te 


‘sonagid 
SOITATIOV Sas Jo onbIs9}e.s UOTeJUITIO 
uOs JURSSTTGeI9 ‘JanuU JospNq UN sWWWOS 


UOTVOTJLIVA Op SUOTFBAIOSQO soNY 


9661 IRIN — epeued np [ei9ugs InsyeolUA np yoddey 


‘yuoWINSOp ayNe 1N0o} ap no ueyd uN,p 
uoneroossy ,| ed uonesedgid vy] & aATNLIOI 
Joud8Ixo ounone jeVJodui0d UT] e 


‘UONPIOOSSY ,| Op SATATION 
sop seiduroosa sjeynses xnv JuRYyoRyeI 
9s step onbrurou0cs Juswaddoyaagp 
ap SINa}VOIpuUl ,p sed jressTUINOJ sU TT 


: IQANOI) op SUOTpUde 

snou snou onb sajuvains suonIpuod say sed 
jreudIdui0d aU pIOdOR J ‘aN UY LVI 
‘IOSOI], Np [IasuoD 

np oiaTpnonsed UoHIpuod ouUN .p Hessise .s 
[J (o0uesyV ,| Jed stumnoj spuoj sop 
JUSWIISLZUA | OP SIO] ‘SIQI9IUT.P S}]JUOS 
SQ] JOA B JURSTA SOOLOOIIP SOUSIT 

sop onbiydde ja a10gry{e a9 .nb s1gTusap 
21109 ap sed j1ea8rx9 .U [] ‘UOT}RIOOSSY .| Op 
SOUATIOV SOP JUSUIDOULUT] IP SUOTIIPUOD So] 
}RSST[QLIS UONLIOOSSY ,[ 9 VOUDSY ,| aU 
NjOUOS UOTINGLUOS ap plosdoR /T Ol'l 


Saeginos suolsenH 


“UOnRIOOSsSY 

yuswdo]aA0q Ye SifeMuslod 

Rl OP SSIANO’ So] SIJLI9A sed suoar .u 
SNON ‘SJURSTFJNS JUSTLIS DATLIVIUT 91199 

INS S3TONUOD Sas Is JOUTUIAIap ap JNIelqGo 
SUID RAR OTTEINvUNWTUIOD UONeIdepe .p 
IANLNIUL | 8 BOUdSY J op UOTedronsed 

RT 9p UOTRSIFLOA ONON ST T 


‘qetdold op yop np weysuen 

9] JUBAR Seq P| Op SUDIG Op JUDUTDAQTUO | 
ap jofns nv sag[nuoj sopmetnbut .p 
ays eT B aJ9Nbud yey ose] 

doI[Og P| onb je; Np ssuJoyur e snou 
a[euoTeU osUdJaq PT ‘966| JotAuel [¢ 

9] INJOUOD 9}9 B JUDA BT “JUSWIOUUOIIAUO | 
& ISVUIWOP NO} ap seo a] suep d3eAoNOU 
np s[qesuodsas yleInowep sfeuoneu 
asudjaq LP] ap sJQ\sIUIW aT ‘sastpNNeI 
11g JUBANOd OU dSeq I Op sJIOR sap 
uoneuslye | mod uonerdossy | & sie[[op 
OP SUOT[IU 9 ap seqo]s UoTUSAQqns 

9UN JOSIOA JIRADP s[eUOTJeU dsUaTIG 

B] OP I1Q\STUTU 9] ‘9INO U_ “9]U9A INI] 


UONLOLIJA op sUONRAIASqO sonny 
—_— SS 


ap INOS Ne DINILIOJUT }Ie}J9 SUONeST[IQOUIWI 
Sop opueyoreul Inova ey onb 

IE] I] INS Sgpuo}j }1e}9 IeT[Op un nod synoe 
Sop 9JUDA BT “IeT[Op uN op onbiyoquiAs 
JUvJUOU J] Inod ‘uoyerMDossy ,| B ISeq 
QuUUdTOUL | Op 91U9A PT WeANoIdde Josa1y, 


NP [fosuoD 2 “C661 [HAk LZ 9 vit 


“SJUBISOI SJIOB SIT SNO} ap jo aseq 
QuUdTOUR | ap ayt]Iqesuodsal e] 19 sjoO.NUOS 
2] “S661 [HAR ja] Np Joydurod & ‘argrusap 
an109 & JURIEFSURT [eq UN UOTeIOOSSY , | 
OOAB JIRUSIS syeUONeU OSUaTaq PT 


ap S1QISTUTU 9] “CHET STEM OE IT = ETT 


“OIQIULIOp 

91199 op Sa}TAT}OR say ‘sue buto ins 
‘IQOURUI] Op UTJe SIeT[OP op suOTT] TU C‘/, 
Op d[PUITXeUI INOTBA OUN .p UONNQINUOS 
ap plosov UN UONPIDOSSY | 99Ak JeNTOUOD 
suas V J “P66] 10k OT OT (am 


“uoneIoossy 

juouidoyoaoq ye SijfeMuslod 

R] op Joptseid ay suWOU & onbruepye 
epeued np onbrumouoss juawaddoyjaasp 
op dUd8Y ,| 9p a[qesuodsa1 

OIISTUTU 9] SUOTIVIOOSSY ,| Op SJUSWIITSOI 
SO] JUSTIOAIA I] DUTUTOD “assodq-a]JOANON 
Ud JIVIONT ING suRS 9}9I90S uo sanjysuo0o 
919 B UONeIDOssy [J ‘sesTTNNeI ang 
juvAnod oseq ke] ap sjnjoe sap siatidoid ap 
S}IOIP Soy Ienboe v 3a uoIssassod ospusid 
B 39 SAVYONO}) SdzI[BO0] Sap onbrurouosg 
juowaddoyjaagp ap saiyiqissod 

SO] JOJUDTIO B ITeISISUOD JepUvU UOS 
‘(UOT]BIOOSSY ,[) UOTJBIOOSSY JUDUIdOTAANG 
ye SI][eMUIOD kJ IOs ‘Teoo] 
juowaddoyjaagp op ourstuesio uN y1e910 
sues V1 “Pool Jorn! 67 97 Il 


“spuoj Sop uoNesT{Iqeiduios 

op 19 JUDWIASeSUD pp ‘UONNALe .p spoul 

9] JURSIOOI SoT[Ie}JaP SPIODNR .P SOULIO} xNe 
xnevoo] JusWdddojaAgp op soulstuesIO sap 
JOOURUT] B DOUDSY ,| JUSTeSTIO}Ne saiTTepoul 
SI) “SOTQISSILUP SoITeIOIJQUNQ Sd] DAP 
UONNGLNUOD op sp1odoe ,p UONIOL]A, | 
Inod oped un ,p JUdWassT]quIe | 


966I IR — epeued np [eriaugs InoyeoILIgA np yloddey g-] 


B SOATILIAI Sa}TEPOU sop }eOduIOS uoneidepe .p syoyya soy jueAndde 

IOSaI], Np [Iasuogd np uoneqodde 7J SOINSOUI op JUSUTASsSTTGeIE | WeUsIdUIOS 

‘SIeT[OP op SUOTT]TU C‘/ OP yea §=— BJT IQusuOdsaI 99D “sdgyono} Sa}TATIIT[OO 

SI][PMUIOZ SOUUDIPeULD Sdd10,J Sap oseg So] JOpre & JUSTIA xNvIIPIJ SIOJJO So] 

PR] Op INJoULIOJ eV] INod gpioooR [eUITXeU JQUUOPIOOD ap saBIeYO 9}9 & (QDUISY ,[) 

JUSMDOULUTJ aT “aseq PB] Op DINJOWIOJ onbnuepe epeued np onbrurou0s9 

ey red sagyono} onbnuepe epeued np uonowoid op souasy ,| ‘onbnuepe 

SONANDIT[OO Sop UdNOS Z| INod sJe][Op op epeues do] SUP SITE} SINjONSeIJUI | 

SUOT]]TU (QE Op oUTWIOS oUN Nod doUa3Y | op UOTONpel op d][oLIE}sTUTUU NU 

op jusWIS0URUTy oy WeANoIdde JoOsoIL, Q[LJUSUIOUIOANOS 9139}e.N)S 

Np [lasuo|D YJ ‘pH6T urnl 6 97 Ol'l P] Op sped a] sued “assodq-a][2ANON 

Ud ‘sT]TeMuIOD souuorpeurs Sd0I04 

"EPEULD XNVJUSUISUIOANOS SIdTAIOS Sop aseg P] ap oiNjouLIaj ey stIdui0S 

39 sorqnd xneaesy, op ererpogursoqur | red A ‘uononpes Iney no ‘anbnuepe epeued 

aINjONseIJUL | op UOTeUgTTE Pp snssaoo1d J] SUP SoITe}T]IUN sjUdUAssT[qe}=e 1das 

9] Jo198 WRAP Jo oseq oUUSTOUR | Op op sINjOUIVy eT WeSuoUUe 

SUdIQ SOP dIIOSIAOId apses ev] JOUUOPIOOS $66] JOLIAg] ap 1o8png oy 61 
ap sjqesuodsoal j1e}9 TeUOTIeU sSUdJOq 

RP] Op sia}STUTW oT “olleyneUNUTUIOS 9}X9}U09 


‘SUOIIDSIIGOUMA XND SUOIJYLON WUD 
Sap PD Ja SUOIID]IDISUI Sap Ualjasua] 0 12 UOYD OI dxa | Y ‘UONDAJSIUIUUPD,] B 

spuof $29 a4aDSUuod JnojANS D UOYDIIOSSY,J ‘aAnosddp juawmaouvuyf uos ap OQ] ‘d 9E 
NO SAD]JOP ap SUONJIU /‘Z UONDIIOSSY,] B ASABA VD aIUaBY 1 ‘apolsad ajjan JuDpUuad 
‘aAalUsap ajJ9d ap UOIIDAAI D] Sasdy SIOW pT and UOIDIIOSSY,] AP SPNANIV,p 

uvjd a] N5al D,U aDUasY TJ ‘Sajqissmupp Ssaynoiavd Snood Sap uoyiuyap V] 

ja anuasy,] 4d siusno{ spuof sap juamaspsua,] Anod sjg4ajul,p Sqjfuoo ap aLaijvU 
Ua SAD1dJIA4LIP Sau8y Sap 4asy1N, p ja 4a40qn]a,p aammiaifauag a] 4nod uoynsyqo,] 
JUAUNUDIOU ‘UO1JNG14JUOI ap PLOIIVD.] SUDP Sasnjoui Spd Juaivja ,U SUONIPUOD 
SajUDjsOdUul, ] “UONDIIOSSY,] JAAD SADIJOP ap SUOIIIU ¢‘Z ap UO1ING1AJUOI ap PAOIID 
UN. Pp Sautsa] xND uawmaauvUlf NV SaigissiUupD sajiAljov Sa] Juaivsas sayjanb unjsoddo 
sduaj ua yqvja snjd uou sod 0,u aja Ja aay aya ap Sajdwmoosa sjp]jNsad 

$a] aououa Juauasnjo sod D,Uu aquasy,] ‘adivInDuNnUWUOD UOIvIdDpD, p aalviziul 
JUN ALANGO UA AAJAU AP APBADYI DJ 12 “PIJDAIN] ING SUDS 9191908 ‘UO1JDIIOSSY 
juaudojaaaq YAVd S1]JVMULOZD VD] paid ANS SIU D aDuAaBY J ‘AaSSOIY-a]JaAnonN] 

U2 ‘SIJDMULOD SAUUAIPDUDI SaI10.4 Sap asog v] ap aamjaulsaf py s14duloo 

& anbyunjiy ppouvZ a] supp asrvjijiuM aAnjonsjsv1ful,] ap Uuoyonpad D] 40d Saayono] 
9141921109 Sa] 4apw anod sansuoa uoynidopv, p saaypiiul sap ajqvsuodsad 

| josapaf asajsiuiul aj isa anbyunyiy ppouvy np anbiuouosa uoyowodd ap a2uasy,T 


dIIeINVUNUIWIOD UOHeIdepe,.p 2AHeLIUI 9UN kB Svat] SUONedNdI0I1g 


uallg.Q uyor : ajqvsuodsas AnavoLfisaA 
UANGPOY +f ‘WM : Julolpp JpAauas ANaDILYIsaA 


anbijuelye 
epeuey np enbiwouoda uoljowoud ap aoueby 4 


UONCOTLIDA Op SUOTBAIOSQO SONY 


966I IR — Bpeuey np [erguas InayeoIJLI9A np Woddey 


“QUIDOUOD 9.19]STUTUT 

np wou 9 snos ‘antdeyo yuasaud 9] 

SUP S}LID9P JUOS S9AIJASqO SUOAR snoU onb 
Seo So] ‘sgururexa sed suoae .u snou aonb 
sjurod op jolns ne suorsnpouod sap Jain & 
JIAJOS sed JUdIVIAP OU SogTeUSIS suonsanb 
SO] “SOOULISUODIIO Sd] SUBP SaTTRSsa09U 
sasnf suoae snou onb sapgooid sonne 

J9 SOSEPUOS So{ 9}JOdUIOD JUO SUONBOTFLIDA 
sou ‘juonbasuos red ‘sanuuoses 
JUSWI[VIQUIS UOTROIJLI9A op sowWIOU xne 


yoddey JUSUIZULIOJUOD SOUTUeXd 99 B aNIdeYO 

3] SUBP SINEyjie yuasoid 9] suep sogyeuUsIs souRVIOduIT p 
wed ajjnu uojuew suonsonb sop ounseyd a | 
jib} SUOAP.U SNOU “OIANIO US SOSTU 


uop aoueyod dN}Q JUSWS|QeUUOSTe! }9 JUSWATQeUdAUOD 
juop Yoau! p juaanad saf[a No seo soy suep 


suoljsanb sap soumUeISOId sap ay1OvOTJO | Ins oddes 
INS SUOI}eEAJaSqO,p alley 19 JoINsoul Inod sarjqeis 919 sed 

JUO_.U sojURsIeJsnes sainpgooid sap e 

djqQuwou 

uleyi20 un.p ay1e) ‘DOUSTIONJO | IP NO sTWIOUOD? | 
op JUeSTFJNs Tonos uN sues sagsuadap 

auyideyo yuaseud 97 919 1U0 JUDIE .p souUTOS sop e 
‘sogqoayye 


WAR So] JUS [IV 9] Soyjanbxne 
Sa[[20 anb surly sone .p ve sagsuadap 
919 JUO JOSIE Pp SOUTUIOS Sop e 


‘S9gSLIONV 9}9 JO ssanqdaqJo 
sasuadgp so] aonb sJomnsse jo nudAOI Np 


UONRITJLIZA 9p SUONRAIOSgO sonny 
eo ees 


Mg[NSaI uoNNIedgl kT op Jo JUSUIDIANODAI 

Np ‘SUONeSTOD sap ddRdTJJO dTO.UOS 

un Jamnsse ‘sorqnd suatq say Jayo.nuo0o 

}0 Jopsresoanes Inod sayuestjjnsur 319 yO 

SogsI[N soinpgood 39 saTsaI say no snus} 
919 sed jUO .U S[aTJUASSa SASSI SOT oe 


‘stnbas JUDWIITeSg] ISa eTIO onbsIO] JOSaI], 
Ne SgsJOA 939 sed JUO.U NO jaTdUIOD NpusI 
adu09 un ,p jafqo J 1ey sed yuo ,u sorpqnd 
SIOTUSP SOP NO dIII[NSaI jd dIplj s1gTUeUT 


oun ,p snus} 939 sed yuo .u sajduoo says e 


: onb 93e}su0d B [I No syuRyoduIT 

SBD SO] SNO} SOUNUIWOD sap dIquIeYD Pv] 
op uoTNUNHe | B oJOd [eINUIS INDJVOTILIOA 
3] onb 981x9 Jp4aUas ANaIDILf1AAA a] 

ANS 10T &| ap (Z)L oydeisesed a] ea | 


“sdoINOsSsal sop UONestundo | 

IP SUONLOTJLIQA SOU Op SINOD Ne NO sayTUS 
SOTNL .P 19 Jk .P S9IQIOOS sap ‘epeueD np 
sajduiod sop suonesuoine XNB Q}IULOJUODS 
OP SUONVOTJLIQA SOU dp Jd SaQTOUPUTy 
SUOTIVOTJLIDA SOU OP S:INOD NV SddAdTOI 

919 UO SagfeUusIs suonsenb seq ‘0ddey 
9] suep sinaypre yzed oy[nu UoNUOU 

}I@J SUOAR .U SnouU JUOPp jd souNTTUOS 

Sap sIquIeYD kL] & JO[eUSIS op ‘SIAR oTOU 

B ‘JUSTAUOD [I Nb soueyIOduIT,.p suonsonb 
ap oyrey) ontdeyo juesgid a] oT 


uoI]}ONpod}U| 


ix 
_ - = 
9 hy 
Se a 
e 


pcaasenennanlin dicta iase synceaeseayone pny, preter gne 
rsa oa ey rt iD a 


vs ; ms mi aaa fae tabet ig, vfebowcin cncutetnileteayin 
a _ ae Ce 
ay earercitbaiee (ateemz bboancart are: 2g Gra ert 
pian aes Out rwaare an, Ger Dertira dine Gone y aaiee® free arte de 
oe RARER GE AGLIEIC IU bouinepgtcs basal ger Recsnte werbeLeny p: poratte ty 


te aoe nt Gif. Se 
a PhGatnnTe Mn hae bouror em FO UO erly Sa OTK STON ee Aes 41. ee 


ir eimmrcene paps Gre oe tAnpesiny 2 ebb NES Ue Gaiectece © ON SEALE Cx 


me | Rotiod suniee ou ini cota We Doe Do UGSTTeDe GFT chatieeenn toe oneucice 
ms Eis pe beam PATNA TPH wT yO | " aaieata A eA ee 
dra. 4% ee ee, 00( DOSE Vacts bas? cones 0 Ct os BGR aie 
ee ay} ys Sepa ASe Sty, wey ger Beveasc DET AG Se Rec WD 2 ce Seether Mais 
? pg v pvoapege bes cyenprots: ne Dice RA oouvi AY Crem rn? Gea? meh age Ae; 
yevere 90 i Bg EAE rath wre jen RAPRMBA Lin? Lope por ye i3))¢ ene eh By ines aore 
4 rae se CAME TID Cl SALE ate 8 STG Ma OY, ORAS ETI fe CUA 


east 


f 
Seals 


a a oerhinn 
ar inthaly a Os pie ee 


» 
ee 
: 


i gerne lege 9 ton were tice LUNA 
Chewy ate an fi mie i Tonge dec Tae SOS eS Yn ew beaker RN pad 


‘ : a 1 je 
2 ain >. ie -_ Par. : - - 
Sa 4, Ee 
tee : ny Wr. 7 
7 pig = - an . was 7 
te Y : J é 
a A = re & a oe 


os 


966] Ie — epeurd np [eigues Ind}eoTJLIZA np yoddey 


‘epeurs ne soyer[rUiey 
SoIONnplj Sop sup SNUA)EP SJOv Pp SIeT[OP 9p SpseI]][TW xnop suIOU ne .p JosueNY | & WosueNy ne 
sanbrdde 9j9 & nuadad a] ans Jodi] ap 107 ke] JOP Vode] eT e yUeNb opryginbut.p syyoul xnotas ape 


“SIBT[OP 9p SUOTT[TU ¢*/, 
op orreinvuNUTWOS UONE\depe .p sANeNIUI oun Pp s9}dWIOdSe s}e}[NS9I Sop ITeTO ouOUY,p anbueUI aT ~—e 


: yins tnb 99 Ans yuUa1I0d suostey snou onb suoneasasqo so] GI 


‘sgururexe sed sue .u snou onb sjutod sap Ins suorsn[ouoo sap Jai nod mAsas ua,s sed op 
au IN9}99] 9J ‘sojuR}JOduN suoNsenb sep JUs[eUBIS sargITNONIed UONBOIJLIQA ap SUONRAIISGO Sd] IS SUIT, rl 


“STUOUODSY || ap JUBSTJJNS Tonos suKs sJoTUSp ap sasuadap 
Sg] Ins onb tsure suonesuoyne xnv 9}WIOJUOD-UOU PT INs 1Uo}I0d sag}TeN suoNsanb say ‘Te1gUas uq ‘SoJo}sTUTU 
Sop soye0e1 xne Jo sasuedgp xne juoyoddes as Inb suonearasqo sop juoWaT[aNIIqey sUIAJUIL aNIdeyo oY eT 


‘S2OINOSSaI Sop UOHesTUNdO J 9p SUONBOIJLI9A SOU Op SINOD Ne NO s9yNUS sone p39 1e1q .P SpIgTOOS 

Sop SJoroueuly S}¥}9 sop “epeuR. np sayduio_D sap suonesiioyne xnv 9}IWOJUOS op SUONBITJLIDA SoU ap 19 sorgioueuly 
SUOMPBOTFLIDA SOU 9p SINOD Ne soUBSsTeUUOD sid suOAR snou JUOp sestogid suoNsonb sap Ans j10ddei suostey 

snou ‘onideyo 39 sued ‘o[quasud UOS sUep JUSOUIANOS np sayTANoR xne SdATJLTOI SUOTSONb sap Ins yUdII0d 

Inb sopnig.p Jo SUOTBOTJLI9A op npuss a}dw09 un yUOUUOP SII UATG No ‘sIot[nonsed sa1g\sturul sap suep sagnioaqyo 
SUOAR SNOU onb sagiSQUI SUONOIFLIDA Sap SIO] SOUT SUOTIEIEISUOD Soy JUSWATTONITGeY JUSALIDEp sanideyo sonne.q 
“yoddes a] suep sJornonsed a[Qs un onof « uoNRoTLIGA op suoneArasqo sony » xne gidesuod ontdeyo oJ rE | 


“SOUNUIUTOS Sop oIQUIBYD) BT 9p UONUSNe | v Sagyiod ang JUsTeIANp ‘stAv UOS ¥v ‘Inb souRVOdUIT,p 
suonsenb saz sjroddei sos suep onpout [1 nb yerguas INIJVIIJUIJA NP IBIX9 [D1GQUAS ANAIDIL AIA a] ANS 10T BT Lal 


S}UBI|IES S]UIOd 


UOITROILOA 
ap SUOIJeAJaSqO Sadjny 
See 


a = bepconasere a _ 8 
nnbet bea pesonstie > Yapi te sor tenqne - ! tts 
> pe burnccsberyse gelyee anton ine ya prey suncrbad Qi iateceyee 

. mong Peaeer Rites Os BCS ty nulos ihn G fag! =i 
C-6U. von om rCetE: TOT et pated ifs 
men "ihe eae a 16H re 
va 


oe : sda = Lene. ge? 


> 


; is SOV fens P Igie ge 1. ek Od is3 
wens 


eas a 
rhe tas ee. uals | 


i fas bie ana 
if nha ik ria cance 
: e <0 


“Foye buamote concen Gh citevga ria ar 


a 1 i ‘“ 
f 


a : ; | tp 

i PaO ATT | bs 
j , . { BEE. 

J 

: 


966L IRIN — epeueg np [eroUas Ina}eoTJLIVA np oddey 


tea 
OCI s[qenqiuod 9] ed sagnqoajjo suoneiadgQ Cu 
gI- ONPUSI JOS OU [66] IP NUDAdI 9] ANs JoduIT.p 
o1gneul Us ogdrionue uoIstogp vy onb yuear sadej9 sayedioutid sap a1ZojouoryD I 
9I-I 1661 9P NusAaL gj Ins joduIT.p sJgneUI Ud sodtonue UOTSIOap eT ap onbruyos) sTToqQr] i 
SI-I NUZASI JJ INs JOdWIT,p sIINeU Ud SRG] US sgUUOp UOTUIdO J PGA 
vI-I C861 9P NUDAII ZI Ins Jodut_p signe Us sodtonue UOTsIDap eI op onbruyos) s{[9qQr] ial 
SII 
CL=t epeUe.) Ne So[eITIWey Satonply sop suep 
SNUSIYP SJHw _P SIe][OP Ip SpseI][TU XNop SUIOUW ne .p JosULIS | e Woysuen ne sonbridde 
919 B NUAAAL 2] ANS JodU1,] ap 10 e| JOP UOSe] eT B yUeNb spmembut .p synour xnatIas 
CI-l epeuey nudsAdy 
g-]| sTTeINvUNWIOS UOTe\depe .p Saneniul oun v saat] suotNednod0e1g 
Sal anbnueyje epeueZd np snbrmo0u0s9 uonowo.ad ap auasV J 
yt uondNnpo.nUy 
aa | syUB]]Ies sJUIOg 
abeg 


SaJaiew sap aqe] 


‘Saausy Saiqnidoy sap uaipouvy InjYsUy,] EP (dSAQD) 2qnd 

ANGJIAS A] SUDP UOIJDIYIAIA VD] Ja 2jIIqvIAdUiod VD] ANS ]1aSuOD a] Avd SaapuduwloIad 
SIULAOU Sa] Juajoadsas Sapoyjau ja sanbyyod say "JDAQUaS ANAIDILIAIA 

np nvaing np sapoyjau xnp ja sanbyyod xnv ‘f110]818.a] JOPUDUL ND JuawMaUsOfuog 
SIPUIUL JJ JUO SUOIIDAISGO SasJND xnD Nay aguuop uo nb suoyvILfisad SaT 


UOIJBOILaA 
3p SUOIJEAIBSGO Sa}NVY 


—— panideyg 


PREY Ox EN |) eSyteNS | A! 
i tify mK ” 
AORN 400A 2. SPHigo or 
ee ifn (2D 2) tj wri 


nA wp BEE, we pean) ee 
pl aceswrtinds ary 9 Age ae oealsrs Panty a eee | — 


a Brig «zee ae 
Prt) ones tages 


nel ty FG esi ap hay Usain alee oo Ret ® rj NPE HE RT ie ey 

= ea SAO DAE ak ei ip GeemayEmy “oo 7 Cage He qagermeaeriany —' 
eae apr PO | MARES! or oN tpylal ere Hy Ret 
> RS ee yD 2 Goan 


RLS RapaAerpccete giv O52 Git pu) Ga $963 <t Bal « wis : mare’ 
toe: Roh Al Salle Recah a vanaag. pei) Ke PW fares) air tila’ aed, : db 


i ' wy ore ire TY < a 
Se ee ig a GUI AD verde ay Senay Wren cy wy be i COG 
a: ae taal "6 gs tomicingl wen lar ee ee a 13 “ oe 


7 7 a d i a ; A be ily > Co *, “ayy 
iro wis 7 Da. aiiawe PLATA | oe Ripe i STAAL ed) Bea V9? ee, ’ a » Chi palee 
Ad ha: ha ied DA) eos KuDtIRR ec sgreni ary nisi, (a seca eve 
Ca hiepteitind sso ME afi « Apia ati porta ss KP es ‘ aot ni i om 
ua = 
> ; in ee far (S47; f g $OHG * *; 
tae ns ilgili re hall Jen@ yng? pe il dad owes ed « a \ nat ae 
a < weave > Y av) aits wu ty, a % —-" ‘ coer ies, 


i? 


seins en seaesiiece. Peary PT) 
nL , ni) | ba ee. 


Hie 


: i Vigheeie ws 


Bae penczce neg es canner susan om are RE a re mS 
it 
a rey tere 108 fo weg:  brals IG TIEDO PAT ON 1 POS AY (a OH PETS Oe Pike SRAM - 
_sauabntie! wor dabiniwese hot. avcigl.deee dace Greens 14 GLa Ors. F oay ape Te onl 7B tS Qhe, 
= iy L oR 
L pedo qe tote qu eos’ cheque eee dey be. jt cy et: on ieee, ® rayvenel's we ete Te ot Ge uyeer 


Sterne Hee beset ore mat atoll alae Ieee « gee yaa we heed nips abe 
RESP OE” tigre: NTS oa cep hve Te Ts a eon oo 5 sided ee? 
Paap aree ae Pe i 2 Do hene GT) Roe bent, | care aoe taser apie Siegwdtes cap t. 2 or ig) 


b.. Baw maa AreA te CONpA ait “pee OY coe TY - nso Sn a 


ses es 


upuile.) 3 


066] IRIN — Bpeued np [eigues ImayeotyL9A np yoddey 


"9SN9INOD ISO IIQI] DUDA Ud SJUSWIBSIPSU SOP 9[QIUOS op spoyjeUT Bs ‘aNO UG ‘saJOI[sUIe 

S2[Q.U0S sap UoHeor|dde ua juR}OUT Ud S}1ajJO SUIOS sap syITeNb vy JaJOT[PWe 19 d9UUR Jed sIe]Jop op suoTTUT Jdas 
OP SETWIOUOSS Sap Jost[eas WOAQId aJQISIUTIY 9] *L66] US UOnRoTdde us emow [I,nb syuowesIpour op soSvjueAe 
SP SONOS 9p SOPpOYIPUT SO] JOJONSWL .P 9ISSI9U BT NUUOSII JUDUTITe9 B [J “SJUAI[D sas B SIaTJo sonbynsoeuneyd 
UOU SIIAIOS SOP I[Q.NUOS op oUIE}SAS UN JULINEISUI Ud SOTUIOUODY Sop asT[eal B SIIISTUTPY 9] ra | 


“SUANS|JOO INI] 9p Ulas Nv jo XN9 ZIYO IUIOUOINe UOSLJ OP SIAIA & Sapre ¥ SJ [I,S NO sarTeTOIaUuaq 

Sop 91Ues BT] INS suoIssnosodgd sap no ve sWWeISOIg 9] aINsoU o[fonb suep Jo1LEA Inod sogudosdde sonbuidwio 
saguuop ap sed asodsip ou a1g\sTUTA] a] JOSIL Np [lasuog ey sed yosaid JUSUITeS9 ‘Q]ULS Op SUIOS Op SUTOSAG 
Sop UonenyeAg.p snsssooid np anideyo ne sounov] sop gadjad suoAe SnoN “OMasod p syuoutared sap UONRIIJLIQA 
op stuaisXs np aynsinod ey yURUIEDUOO JOSPI], Np [IsuOZ oI Jed SaT[qeI9 Soy[BpOW xNe IULIOJUOD JUDUTOIgNUS 

Sed ]s9,8 ou [I ‘sjuR}RqUIOD suaToUe sap oTWIOUOINe | INod SUIUWIVISOIG NP SJANIO Ud ISTU LT OP IOUDTONJO | 
JoIONsue Inod snSuoo sjuswesueyo sap Joie \sur,p sudanuo v epeueyD sjuryequiog susiouy ,nbsi0'7J La 


(a}INs) syueljies sjulod — Z| auideyg 


9661 IRIN — Bpeury np [e19ues InaevoTJOA np Woddey 0Z 


“Sple}OI Sop JUDANOS JUOSNdOR JUSTIOUUONOUO]J Np UdIeXA Pp sNssad01Id So] 19 JOIOUBUT 9OIOIOXA | OP 
JUSUIIOUDUIUIOD 9] soide sajuasaid JUDANOS JUOS sJaspnq soj ‘souNde] Sop JUdLJOdWIOS JUOWISUUOTJOUO] op sojUa}US 
SO] ‘SOUSTIOLJEp JOS $]ULJOV.INUOD SJUDUTASST[QUI9 SO] SUBP SasUddSIP SUTOS SOP NOD NP I[Q.UOD ap Sapoy}euUl sa] 
‘JUDUASST[QuI9 US ddINP INSU] ap suros sap suenb ve] JURUTIOUOD sTeUOTILU DATIOIIIP 9UNONL 9}STxX9 ,U [] ra | 


‘Q1UeS ap SUTOS Op SUOTIe}soId xne 
DUTIGISSTUUPR | JOUTULIDJ9p Inod sayuRsTejsHes s[O.NUOS op sopoyjauU op ssodsip sJgISTUT| 9] “STAB AOU V STI 


"91s Op 
SUIOS dp suONRIsaId sao Inod oguur Jed sJe][Op op SUOTT[IW YG UOITAUS OSIOA [IND I[qeI9 B DIQISTUT|Y 9] ‘NUDAI IN] 
OP UOSTRI UD SOTQISSTUIPK SJUR}eEQUIOD SUdTOUL XNv SAgSIOA 9]URS IP SUIOS op SUOTEISOId SoUTe}LIOD op sIT]Iqesuodsar 
PR] op jofns ne saoutaoid soureysao jo epeuRd sjuryequIOD susToUY onUS sInpsod Inb surgyqoid un ev A [J wat 


‘JNOalgo 399 Jojodsal v eIPUSTAIEd JIQ}STUTPL 
J] IS USAR] & JOIfLIQA SUOPpUd}US SNON ‘ddUR}sUI oIQTUNOId Ud sagjUaseid UOISUad ap SopuBUOp Sop JUSWA}TE.} 
OP SILTEP SO ‘SIOU! 6 & QI Op ‘SUB XNOp IOI,.p oIINpPal B gseSUD JSO.S EPeULD sjUL}eqUIOD sudIOUYy erg | 


“SIDISTUTPAT NP soumuuessoId soy 
INS sJUDUTSZULYD S99 op SUOTSSNdJadaI Sa] 19 9IULS OP SUTOS Op dIQeUI Ud s]UST[D op UoNe[Ndod es ap 
sINjNJ SUIOSAQ SOT SUP s}USUTAZULYO Sa] oUIJODUOD INb 99 us suoTstAgId op T[quig sed v.U oIQISIUTJY OTe 


“epeurd sjuRyequiog susIoUy 
red sovJoJJoO 9]ULS ap SUIOS op sUONLIsoId xnv soTqISSTUIpe 1g ,P 19 SJUSI[D Sop JMMaAsp ap sayqudaosns 
JUOS SAQUIIL SAOIOJ SOP SOIQUIOUT Jd SJUR}eQUIOD sUSTOUB XNOIqUIOU Op ‘ano UG “g]URS Op SUTOS Sop 
anideyo ne IUdA & d[QUST]O vs op souRVOdUIT | JOAgId op yUo}ouNIEd Inj Inb soguuop ap ned saneiT e 


: SQJULAINS SUOSTLI Soy INOd ‘9}URS OP SUTOS Op dIQTVUI US SUTIO 
sds op SININy suUTOSaq xne oIpuode k JURSIA [egoTs ULI UN jUIOd nv stu sJOOUA sed BU aIQISTUTI O°] rie al | 


‘08 SoguUP sop UT] FT B INOVINS ‘UONRTJUT,| op jo SoseJULAR Sap UOTBOTJIUOG 

Bl Op ‘sourUeIZOId XNv SITIGISSTUIPR ,p SOIQITIO Sop JUDWIOSSISIE[A | op ‘sooutAosd say 99Av soUd}9duIOO ap suoNsonb 
ap uostes ud QOT “d 6p] Op sJUEWSNe JoJJo Ud JUO s]NOd sa’q ‘aprdes snjd vy souRsstosO eT NUUOD B INb INIa0 

JSo Q]ULS Op SUIOS sap owUeIZOId 9] ‘soguUR SOIQTUJOP XIP Sop SINOD NY “9JULS 9p SUTOS xNv jo NUDADI Np UITINOS ne 
‘suorsuad xne ye] UO dIAISTUTAY Np ssoueUTy souMeIZOId xnedrourid soy] ‘epeued sjueNequIOD suslouy Jed sy1ajjo 
soumUeIZOId sop s10A Jed syUR}EQUIOD SUdTOUR Sas SIDAUD SUOTRSI[GO Sos JUOUUUULJOU jI]dwoI epeUueD oT] Eva | 


SJUBI[IES SJUIOd — Z| Suydeyo 


uaug,.Q uyor : ajqosuodsad AnaqvoifisaA 
UANGPDY J WM ¢ Juiolpy Josauas ANaIDILYIsIaA 


9]UeS 8p SUIOS S87 


pepeuesy sjue}equioy suaiouy 


61 966] IRI — epeuRd np [eigues Ina}edTJLIVA np woddey 


‘a}dUI09 aIPUaT ap UOHRSI[QO | J9OIOJUAI ap Jo SAMAK Soy JaII[dums op souyprqrssod sop 9}STX9 [I NO sna}oas 
Sop suonbrpur snou “oddes anou sueg ‘uonesnoooine ,p oult3er np 9139qUT,| muoyUTeUT INDA uO | IS d][aNUASsa 
189 a][9 ‘snd aq ‘sajueyodumt sayeosty say990I ap ao1nos ong Inad eosty JUOUIAITAY | Op uonusagid ey] ell 


‘soqueyodurt }9duit.p suonesyod sajf[aanou op guTeNUS JUO ‘snpuar soUTUOS snou snou no 
JOMISTP OP XNBIING So] SULP “TEOSTF JUSUISIIAD | 2.UOD 9}IN] OP SPTANIV sor] “ILINODAL K.P So[qeNquNUOd soy Jopensstp 
19 [POST JUSWIIIAD .P S9pgoo1d say ameqQuIod Inod sastogid soayENtul sop sudanuo v o1Q\STUTP oJ TIL 


"SUOTILST}OD Sa]JIANOU 9p SIe[[OP op suOTT[IW COE 

9P JUOUTOSSTIGPI | “9661-S66] UP ‘outeNUS oUTUTeISOId 9] aonb 99 K puanL,s dIQISTUTJ] 9] ‘OTBOSTJ QNoISSe J 
MOUTOSNOLTIOS JOSBUSUT AP BNUTUOD 19-InJao onb juanbrpur [eosty WOUTOITAY | oUOS JaNNY INod epeueg NUDADY 

op suWeIS01d Np syeINsgI Soy ‘stozoynoy, ‘nuUOD sed jso,U NUDAAI 3] Ins 1OdUIT,| 9p JUDWD}TAY,| OP NOD aT lial 


SJUBIIIES S}UIOd — |} dIdeyy 


uly] A4iDg : ajqosuodsad anajwoyfisaA 
Olu plyvys - Juiolpy Jpsguas AnaywoIfisaA 


NUSA9J 9] INS 
}QwI,| 8p JUSWATIAS,| 84]U0D ayn] eF 


epeued nusArsy 
a 


90661 IRIN — PpeuedD Np [ei9ues InayeoTsLIGA Np yoddey 8] 


‘uoneideprss 

ap souUreISOId sa] SUBP JUSTTASSNSaAUT UOS oNbIsa}eNs UOSe} op Jo}daIJVII ap 19 JoN[BAdeI Op dInsouI Us sed 
}S9_.U [I ‘SJUOWIDUSTIaSUdI S90 suRS ‘UONLIdepeal op soUTUeIZOId sad op SadeIURAR SJ 39 INOS 9] ANS sjo[dur0o snjd 
SJUSUIOUSTOSUdI Op UTOSAq BIN BdIAIG JI ‘aIqr[INbs 399 oIpUTaHe INOg “9ILINDYs 9]N0} Ud UOTIVISOIUTOI Op [eqoys 
ynoalqo naj ap UOMOUO] Ud sddINOSSsaI So] Ja}OTe Pp UOSeJ dINo|[Iow! BI JOUTWID}Op op so Inb 99 nod sinotigdns 
solreuuonses sap Ved ev] op souryista op onbuew un ev X [I ‘stojaynoy, ‘UONUSAIOJUT Pp soumUeISOId op [TeyUDAg 
QBILT UN IDIOGL]E & ISSNII & [I JO B[VIOOS UONVISOIUTII LI INS SOIIAIOV SOS OX¥ [OUUONIILOD IOIAIVE 9] S01 


(ayINs) sjuRlIes S}UIOg — O} auydeyD 


LI 9661 IRIN — epeued np [eigugs InaeoTLIJA np woddey 


‘saoInossal sop uoNTWedgd vy Ins sgiqiinbs UoNNyOs oun & JOALe Ud ynad ou {] ‘syuenburjap 

sap aytyiqeAoydura [ ap uonsanb vy Ans Jayouad as nod aUa19YOO sISg}eNs op sed BU DdIAIOG VT “aITANDOTJOO 

B] 91sg}UTaI UO sjI_nb stoj oun ‘peavey np JoAnoN v syuenburysp sey Jopre inod nad say ajsaJ ua [I,nb a10s ap 
‘SJUDWASST[eI9 SO] INS Sd9XB JOS IdIAJag Np siTIqeAoydurs .p soururesso1d xnv sagioesuod saomossal sap QOT ‘d 66 
op snd “uednosogd yey anny ‘soVajjo JuaTe}9 Ing] Inb saouaj}gdwio09 sop Ulosaq JUUIAMTeSsodgu sed JUOTeAR .U 
NVOUOD 8 sgoaye syuenburjap sap gnrour vy ‘ano ug “uroyd sduray e juerpmie Jed ¢ QOS / UOIAUD & DATA 189, 
a[JouUOrssajod UONPULIOJ PT op 39 FVONpS owTWIeIZOId Np INO 9] onb stpue “GEG [—f66] Ud ‘uro]d sdura) ¥v januue 
aysod red ¢ 000 €1 91009 UO NWDAOO op ules nv stojdurs seq ‘soguur sasgrusop ayenb sap sinoo ne sreyjop 

op SUOTT[IUT QO] op snjd Hnopsua k [I :edtAJag Np xnayNo9 snyd 9] UoNedepeas op ourueIZOId 9] JS9 (TeI9OTeO NoTTTUL 
Ud [OLsnpul Jolpoye UN) NYOUOD ‘s[euuontpe.y sourureiso1d sap uoNses ve] suep sounory sap Issne v A TT ror 


‘SgI9qI] S[anxes syuenburfap sap saynyoos sap 

uonuaagsd op ourtuessoid un ev uonedionsed op opuewap et ap QI “d sg &,nb ospuodas ynod ou IOIAINS 9] ‘ayjamjoe 
aINIY J Y “SUANITJOO BY v JUoWassI]quig | ap asessed Ang] ap sJoy sjuenburyap soy Japre ustq snod sourwesZo1d 

ap ayquuasua un Tqvi9 JusWOUTe]d a1ooua sed BU BdIAJOg a] anb yso suNde] aNNe oU_ ‘sJUeNbUTTEP C00 ¢ 

inod sagsuadgp juos jua}se1 nb EQ] “d pg soy anb ssoye syuenburyep OQ] owen eB JWOAIES sogsuadgp soumuios sap 
OOT ‘d Of ‘aruewoorxo} | op jusWA}Ie.N op sourtueAZOId sa] ouoOU0N INnb 90 ug ‘sJUeNbuTTEp 008 | Inod sagsuadap 
Juos jua}saI INnb QOT “d ¢g sey onb ssoje syuenburjgp QZ Joye & UOAIOS sjanxas sjuenburjap soy inod sourwess01d 
XB S99IOBSUOD SOUIUOS Sap QO] “d LT ‘Isury ‘}N09 Ing] JUaTFNSNf snua}qo sy}eyNs9I say onb onbiput mb asAyeue 
aun _p Jesodsip sues no ulosaq UO ua Inb sjuenburfep xne sTUINOJ JUOIES sjUOWOITER so] onb oouvinsse | MOAR 

SUBS UONUIATOIUT P SddINOSsel op QUUOTIOdoIdsIp s1quIOU UN s}eNUOD XNap B BIORSUOD 9dIAIAS OT ‘UOTUDIAIOJUT Pp 
SouMtUeISOId XN S9IOBSUOD SIL[[OP Op SUOTT]IW pE sap UON]seS eT suep SOUND] SOP 9]2]SUOD SUOAR SNON COr 


‘oyeuOneUIO}UI 

e[f9Y99.[ B SNUUOSAI JUOs suN-sonbjanb juop soumwess0sd op ayueuUOTssaidurt otas ouN NSU B JOUUOT}Da1I09 
S9IAIOS 97] [RLOOSTUL JUSWI}IOAWIOD 9] J9 STUPLOSTXO} PJ ‘a[JONXas BOURIAYP PI SUTUIOD sIND}9KJ Sop Ins yua1I0d 

SI] JSUUONIALOS BdIAJ9g Np UoNedepvas op soaTIejUd} Sap aud Ne JOS UONUIAIOVUT ,p sourueIsoId SOT ‘QQ6] 
sindaq ‘aiyiqeAojdura | ins soxe yUOs speLNsnpur sraryaxe say 19 g][OUUOIssajoid UONeULIO} eT ‘UOTeONp?Y,| SUTUIOD 
speniiqey soumumersold soq “[AUTWILID JUSUIA}IOdWIOD Ne JWENgLNnUOD nb SIND}OVJ Sa] INs soxe soumuuessoId sap 

B SIe][OP op prel] [MU UN_p Januur Jospnq os ap Usd Inod das UO.MAUD a1ORSUOD [AUUOTIELIO9 DAIIS I] ZO 


‘QHoqI] Us asTU Ins] sgide suonNovsyUT soTfaAnou 

2P STUMLUOS JUO sjuenbuTTap sap COT ‘d gz ‘assed 9] sed ‘anb areprsuos uo .nbsio] JNVOIUSIS yUoUTSJETTNONIed 

1SO PJO) “JepueUT UOS op ojUR}OduMT snqd ev] onsed eT aJUasgidar a[OI 99 onb yuoUUAT}NOS sInortsnid ‘mof un saraqry 
ang Jed juoMuly xneigp9y siatouaytuad say suep SoIgoIBOUT JUSWSTToNIOR sjueNbur[sp COO PI Sop Wednyd vy surwI0|D 
‘Opies BS snos JUos Inb sjuenburfap soy 19\depeas v onsed us 91SISUOD [IUUONDILIOS IOIAIOS Np Jepueul oT Ol 


SJUBTIIES SJUIOd — 0} auyideyD 


UIDIJIAg PlADG : ajqvsuodsad AnajvIYisaA 
SOpDliDg DAD : ajulolpY ajpsauas arava A 


sjuenbuljap sep 
uolje}depesi ep sowweboid seq 


BPBURD [@UUOI}IAII0D BdIAJaS 
See 


9661 Te — epeurd np [eigugs InajeorLIVA np woddey 9] 


‘oletoljgueq syediourid ey] 3so aLNsnput,| jUOp 

SOIAIOS “SOIAIOS NP UOHRIsSaId ap xnv}0} $1N09 sap QOT ‘d 09 anb sutow aiednogu [I “UaWa\Uasard ‘anb 3s9 ‘stojayno} 
‘anole apnyginbur a.ON “9ULIO} 1INOD & saIB0Id Sop Ie] B dIQISTUTJA 2] ‘SINOD sap JUdWIEIANODAI op ajuoredsuen 

Jo g[dnos afnuLoJ sun .p juTod ne astur vy anb Isure sameroryauaq sep UONeOTNUSpI | “XNeUOTeUIO}UT soyoIeUl 

XB SQ008 | 19 SESNPUL,| op sANHaduo9 ey ap UaHUTeUL JJ JU9}J0duIOD onb say[noyJIp sap a}dw109 JusN) UO, | IS 
‘OWMUUBISOIg NB saNgLNIe spuoj sop UOTONpEs o}I0] LT JasusduIOD op UTFe saNne pe BIPUII .S [E19 ‘sTeIy suRS STUINO] 
assed 9] suep Juarey9 Inb saorasas sureyioo nod ginejsut 919 & sINOD sap JUOUTAIANODEI op JUIISaI aT 9°6 


"SQOINOSSAI 9p JENIOe NKIATU 9] JoULIOd Inq anb 99 ap a}duroo sua} sues 

SoINjNy SPTATOV sas Joyrueld ¥ Jo oWMWUIZOIg 9] JONsTUTWIpe B ONUTUOD [I Nb surour sed 9}S91 US ,U JI sTeUT ‘sINOD 
SUTP}T99 JOWAD B NO IINp9I B JULSTA soInsow sop snbrydde v oseisturyy a] ‘oguue Jed ssrey[op op suoryru OO] ep 
snjd 9yn09 39 (a11OyeIOgGR] Op saAojdura 39 sima}dadsur ‘sorreuUONsea3) sgXojdwia QO] | op ejep-ne.p admbe 

oun OsTTIGOU! xneUITUR Sap 9]URS op Jo xnvI9B9A Sap UOT}Se}01d op sUTUILIZOIg NP 9JANIO Ud IST PT S'°6 


“SOARIS SUOTILISIJUT XN 39 SOIPeTeUT 

op Sopquieyy xnv sdvj ore Inod arystuly 9] sud v nb suakour soz anb tsure Jags 39 nAgid yuouopual JJ Jon{eAg,p ja 
erpusdwios ap semejuowopied xne amjoutiod yuassind mb a}0urdons uoMeUTIOJUT,p No JWOWIOpUdI op SINd}eoIpur,p 
nod I[qeI9 UO saIped S9J ‘sIOJaNO], ‘So[quINsoUT sourIa} US spuTLIdxs 919 JNO oUTUTeIZOIg Np synoalqo say t'6 


‘sorqruodsip saoinossa1 

So] oyeundo uodej ap gst[nn v [I,nb saNUOUWPEp no ouTUTeIZOIg Np a]quIasud | INod sgyOLId so] Iqeye “ayTARIs 

INI] UOJAS SIosuKp soy Josse[O Inod o[qely JUSWUNESTJNS UONeUOJUT ouN,p sed asodsrp ou araystUTPY a] onb }INSUD Ss 
I] ‘suoneyodur xne sag] spyATOv So] ANS JW99Ie | aIMOUT op asse0 [I,s onb gyooiddes uae uN suep oie} a] eLINOd 
ou [I }9 “UOHeJUOW|saI eT Jed sagsta sonbrxo} ssouR)sqns sap Jo soqsoy SasTRANeUI Sop ‘soIpETeUT sop uonenyis 

ey onb snyd wou ‘souuarpeued sayeumtue jo soyej939A saoIMOssad sj ns yUased Inb saseuaUL soy ‘onbryAyeue [no 439 
OP JULAIOS 9S Ud “gn[BAg aJODUN sed vu [] ‘SoNbsII sap UOTseS op oJQNeUI UD SUOISIOap Sop sIpuaid v Japre J Inod 
anbijnuatos aseq oun }UINOJ In] snssadoid a-_ “outeWIOp 99 sUep IIT] op Joyo uN oUTUIOD Q[PUOTJLUIIIUT JTJOYOI ,| 

B NUUOdaI 919 o1INs vy Jed k [I fanboy] Inod ‘sonbsti sop uonenyeag,p snssaoosd un siogeya v d1Q\STUIP V7] €°6 


"S9OINOSSOI SOS Bp SUOISSeIduIOD sayuRyIOduT xnv yuRIsN{e s Ud INO} 9IURS BT sp UONDa}01Id P| op 

a[PIoIoUUOS sysoidde Uos JostuJapoUl Inod safquigpIsuod sjIOJJo sop Tdwoooe v [I ‘sdw9} surgUI UN sUR “saTa0Ep 
B [nb siasuep so] aQUOS JUSWpIdel [Sear [I Jo suOTe}Odunt say yUaIIOdUIOD onb onbstr oy INIVAY dIDSIUTJY 

9J ‘UoHenis 9999 Jon[ed Inog ‘epeuedD 9 Inod onbstz 9] Wo190e Inb 99 ‘suoneyodum sayjaanou ap 1a\dasoevp 
jureUOD snjd Ua snqd sp OA as aIISTUTY 9] ‘aTeIPUOUT a[OOLISe SIWIOUODY | 9p UOTNIOAY | DAV 6 


‘uolyeyIodxa .p 

SPYOIBU XNB $Qddv | JTUUTeUT INOd s[[aUasse jso o[RUOTEUIOIUT aT[OYDE,[ B SaTeUNTUR 19 SoTVI9S9A SsddINOSsaI 

SOS OP OITEIIUBS 819 UO NP BdURSSTRUUODAI BT *SIOTSAIOJ 19 SaTOOLIse syinpold Ua sIe][Op op sprer[Tuw | op 

snjd mod oguue onbeys sJa}10dxa ,p gytoedeo es suep jueyodurt soi un gnof ve sagaaya JUSTOS SoITe}IURS SOULIOU Sas 
onb Jo xnejg39A sap soysered op ja soTeumTue sorpeyeur op jduroxe JUOUOATIEIOI 110s epeury 9] onb jIey oT 16 


SJUBIIIES SJUIOd — § Biydeyy 


uolye}UaWa|He, Je UONOedsuI 
- xnewiue sop o]1UeS suNUuUy, SDjsnog : ajqosuodsaa AnavoifiaA 
; 4 8UNOX UOG : JuIOlpv JvAQuas AnajvIIfLALaA 
e| 19 xNejaHen sap uo!10e}01d &7 


epeuey 


aslejuaW|eOJHy 39 ainyjnoby 


9661 IRIN — BpeuedD np [eisugs nayeoytiga np yoddey 


‘SQTUIOUOD ,P 9SeIURALP JOST[eaI op ‘sed SUTe}I99 sUep ‘a[qIssod 
319 yeINe II ‘sa}oedsaI 919 UO sJo3pNq soy 9 SUOTRSTIOINe soy Is ougUI ‘onb g}ou UTJUS suOAR SNON 68 


"$1NOO sep UOTRUOUUSNe 

oun aUTeNUD 19 JOIWJOp 99 op aiTTIqesuodsal eT snug & INb 9d ‘sg[NULIOJ [PU JWOIe}D dJOOITYOIE | Inod 

sjanjoruos syuauINoop say onb jo ‘sanbijgnd saoupuyf sap UOISAS VD] ANS 107T ¥] ap SuoTIsodstp xne JUsTAUOD INb 
20 ‘XNBABI] SP INgap I] JRA $}v.11UOD sap Ins saked 919 JUO sjuRJUOUT sap onb 9}0u JUDUIITeS9 SUOAR SNON 8 


‘Q[QNSWIUII | op [BUUOTIOUOJ 
a1Q}0RIed J] 9[qQuIOU USL} op sIOT[OWe ‘STAR o}OU B ‘sed BLU ‘SIET[OP Op UOT[IW f°] UOITAUS Pp aireyuowg|ddns jnoo 
un v 39 joford np s98seyo odinbg J sed sgumtdxa saynop so] gIs[eUl ‘asvI9 OUWQIXIS Np uvyd np ostidal eT] L’8 


‘sjyoford 

dP SUOTBSTIOINE Sas INS 9dIOXO JOSAIL, NP [losuo_ 2] onb sjgyUO np gyoKOZa | Ins 39 sjoford xnw sg19esuod 
sJo8png say saynoyeo JWos JOP UOSse] v| Ins suoaZoLIA}UT snou snou ‘gAnosdde jospnq np yuewossedgp ye A 

[nb sues soyenrur suorstagsd so] $9179 SaNQUI YOO €1 op essedap aiiNsuOS aynIq atoTJodns e| aonb so1eg 9°8 


‘SOWATOR SINT & 19 SYOS np sekojdwo xne gS9joid [reaey op noryfrwu 
un amnoosd ayqnowrunt [aanou gj anb ojos us oITej ¥ Issnas ev yofoid np sgsseyo adinbg J anb suoAoJo snoN sg 


‘g[qnowrut | ap yuednd90 J ‘SYOS a enol ve .nb 
juerapuodaid a]Q1 9] 919 B OUSSNEI Op sINdJOvJ Sap UL) ‘SoTeTTUNIS UONONSUOD op sjafod sone .p Mod sayHN Jap9AQI 
Qs Juatelinod souatiedxa | ap Sagi) suodgy sat 19 Jofosd np oyIssngJ RT B gNqLNUOS JUO INb sin9}oRJ Sa] rs 


‘sjuednos0 sop spauUONoUO]J sutosaq xne puodgl sfqnowut | ‘sntd oq “yeSpnq 2] 
g100dsal B 19 SnAgid sIe[ap So] SUBP SUTULIO) 9)9 B [] “oTQUIOSUA | SURP 9193 UdIq 919 B JoOLoId 9] ‘STAR OOU Y ¢'8 


“syUdIETJIP xNVdOT XIP 
SUBP JUSIVUSU BMLIIO,P [BUOISEI NeaINg ZI 19 sTe.NUSS UONeNsTUTUIpe J aonb sgyTANOK Sz] ‘nary [Nes un ud “JadnoIsaI 
ap SYOS ne stud v ‘oGG] Ud ‘oTeIUSO UOTVNSTUTUUPR | IP I[QNOUIWT [OANOU Np UONONIsUOd kL] Op UTJ VT 78 


‘Inassaoepeid Uos ‘epeuRD np sfeAOI dLIOWIepUusyH 
] Op ayLInogs ap BOIAIAS UdTOUR | Op xNdo ap sasedgs JUSTeIAS xNedOT So] JOP (DV) s[eNUSS UONeNsturupe 
oUN Pp ‘pg6] Us UOTIeEIO vs sIndap ‘UIOSaq RAR (SYDS) aILINIS op JUDUIOUBTASUdI NP USIpeUed DOIAIVS 9] ls 


SJUB|[IES SJUIOd — g ayIdeyd 


9|QNSwWuwuI,| 8P UO!ONSUOD ey AGHOODES z 


9}INDES ap JUBWsUbIasUaL 
Np UsIPeUeD BdIAIaS 


9661 IRIN — epeueg np [er9uas InoyeoytIgA np woddey rl 


‘sonbrdxout onsed opuvs3 ua ouop jua}sol ‘sagst{iqeduros 

soyiod say onb 1sure ‘ssuuonuouisns sjusura}snfer sary ‘stnbor syneornsnf syuaumnsop sap o1quiou U0q 

IANO] op s[qudvour 919 & SIISIUTPY IJ ‘UONRITFUGA JOU gnOoJfo SUOAR SNOU NO JUSWOU Ne *stofoyNo|, “sse[[Op 
Op SUOT[IU ¢] ap soyted sap gst[Iqeidwios JUSWATLS9 ¥ [J “SY9O}S SEP SOPTOS Sop INs[BA LT SIT[OP ap SUOTT[TU 19 
dp impel v [I ‘s}uourasnfer sop reg ‘soeIpmig suoae snou onb xed ey op usHUIeW op suorsstur sop Indde | B sIe[[op 
op SUOTT]TU ZZQ op NdTeA oUN,p JUoWdinbg | op 19 sjuouToUUOTsTAOIdde sop sIpsdxo v oIQISTUTA| 9] Cty 


‘Q]SO1I9} 9010, B] ap sfouuoneiado suvyd soz sed stnbos ustNos 9y ITWINOJ 
einod ou 9}0fo1d ouSedureo ap endoy,| onb yuonbrpur org stu Np [OIpsu UdTjNOs op sued soy] e 


‘sadnon xnv sagijuod suorsstu say onbstJ op ofquidaoor nevoATU 

UN & JaTUNT] op JUsTeYySeduIe sapurTY saTNOTYYA Sas op SonUUOD UaIq soUNoeT Set onb nfouod kv sfeuONeU 

gsuayaq] P| op sJeIstUTU 9] ‘S66 UA “xIed op uOrssTUI Us sagAo[dgp sednon xne juowiodinbe .p 
WINUITUTU NSATU UN ITUINOJ Inod sjuvyOduI sINUTU sIQIUJOp op s}eyor sop oMIej NyeSe[] ¢ 


nal] no sed JuO0 ,U soguue soIgIUIEp sop 

sinoo ne snagid din319AUd IPUIOU Vp sdd1OJOX9 XnoiquioU aq “xted ek] op USTUTeUT Ne sagt] Soyo") 

Sas Op UOSTeI Ud JUTONSAI snfd dnooneag sordJoxe un sed govjduras 9i9 & JNSNeYXo snjd 9] VoONAsUT .p 
QO1DIOX9 UOS “dANOaTJ[OO UOTONsUT .p sueyd sos UONDYXO ¥ JUSWTeIO} Sed JOU OU N}SOLIO} 9dIOJ VT 


: soqUaTeAATOd Sa0JOJ Sap JonITsUOd & aI]S9II9} 9910,j BP] op opnynde | ap 
1so Inb 99 Inod sauig[qosd SaAvI3 op INJOSsos je] B SogUUL SaIQIUJOp sop xted ve] op USTUTBUL Np 9dIAIOS 9] tL 


‘xred op UOISSTU Ud S9}STAJOS9I Op UONesTTNN,| B e09ua yuasod as onbyesd 

IPJO,,P SUOTOLNSEI sosnaiquiou oq “xted ev] op UsTUTeUT op SUOTOUO] ap s9sseYoIns 919 sed qUO ,U sore} TT 

sg] ‘seston qusura10J snqd say JouUOsIod np soIQUISUI Sap SdTIO39}eO Soy SUBP SUIQUI ‘s[qUesUA | SUB ‘mof a0 kB 
SOUTUTU 939 JUO sUOTEJUOWISNR some so] “xted ve] op UsTUTeUT Ne sg}eqTe B yUNUIDUIOANOS 9] onb sosreyuoursyddns 
sIeP[OS YOO € S9P 009 T B INeAINbe ejaD ‘ome ][TU JouUOSIed Np UOTONp9l LT 9p sAped 9] SULP SeLIBd9 

919 JUSTeINe JUOUTANNNe INb syepjos sop suseduKd ap SJSaJJa SOS SULP JAIOSUOD B 91}S9II9} BIO] B’T CL 


‘sjustuoSeSud sorNe ,p Ins JUoWSTO[dep UN ,p BUEPTOUT | B 19 UOISSTU UN Pp SADUISTXE SO] Jayoodsal op 
SOUUDIPLURD S910, Sop g}10ede9 BI & ‘a[qUIOSUS UOs SUPP UOISSTU 9UN Pp o}ISSN9J ap siqeqoid ev] e yey & Inb 99 ua 
UONLNILAY,P SANTEPOUT Sas JIOT[PWIL SIOJaINO} & OIQ\STUT] OJ “SUOISSTUI Sop JoUUOSIed Np dAgIe1 PI 10 yuawstojdap 
9] auIs9U0D Inb 99 Ua sassa[qie} Sop gUaseId JUO UONROTFIUETA ap SOITAN|IV SOG “SPLOS SQI} STR[9P Sop SULP JUSANOS 
‘QpInyy o}X0}U09 UN sup xted eT op UsTUTeU Op sUOISsTW soy aIsTURTd sfeuOTJeU osUEJaq] PI OP SIQISTUTU I] reef b 


‘JUOWIONOASP ap 19 SINOARIG OP So}de ,_p sognbseUl 919 
JUO SUOISSTUT Sing] 19 xted eT Op USTUTEUT NP IN9}99s 9] SUBP B[QRIAUS OYSIJ 9UN JUO SOUUIIPeULD SBdIOJ SO"] ie 


S]UBIIIES SJUIOd — J aIdeyd 


YOANSDY Aaja : ajqosuodsas AnavofisaA 
Apayoy piavg ; juiolpp jpsauas analvolfisaA 


gjeuoljeu asudjeq 
— xied bj ap UsI}UIEW 97 


SETS ES ED SPR SEY DI ET TS SL ES ESET SR Te ES 


9661 IRIN — BpeuRD np [eiguss InayeoTyLI9A np yoddey 


‘ANO.[ ® Soue[oar syudtasInoquias 
Sop UOHP.ASTUTUpR | TULaJeI ano Ud espn; [J “xted e] sp uaUTeU op suonesodo say yuRUIaOU0S 
SUOISIS9pP So] AsULARP JOIIEIOS ,.p JUsTeMMOULIAd sonID.e i[iqesuodsar oun jo douaredsuen sup 3°9 


"SogunjoRy SOUTUTOS 

SeP (NO.| Jed uonedacor | & 39 soanvornsnl soogid xne jo sornjoej xne uUataIANOdel ap UOTJOUOJ kT B epeurD 
9] Jed ogpsoooe sytord e| & “QNO,1 ep SHolOse op uoHenys ee aed apueis ud UdUUAT) syUStOSINOquIAI 
Sop supidei ey 19 yuRJUOU 9] ‘QGH] sIEUT Tg Ne snp jUsTe}9 IN] SIeT[Op ap SUOT][IU 7G JURSTTe}O} SJUSWASINOQuIOI 
S°d “NNO.I 9P Sze[]Op ap suorjrur g¢Z g1Anooal & epeURD 9] ‘soguue salgtutop burs sap sinoo ny L’9 


‘SoINotIg\ue suoTedionsed ap so] asinboe soustigdxa | ap juououtatd Jay1J01d sed ou ap onbsi 
9] JUSUIsNe oUNde] 99D ‘o1aSuENE enbntod ap onbydo aun suep xred e] op uonureu op uonesedo oun.p « soon 
SUOSI] » Sd] JOUTULID}Ep no SUONENTPAD Sop Janjoajjo Inod at[qeig spoyjgur ap sed apassod ou SIQISTUIPY OT] 9°9 


(9}INS) syueljIes syulog — 9 anideyy 


966] IRIN — Bpeurd np [esoues InoyeoTJLIEA np yoddey ZI 


‘uOTRUSWISNL 91190 & JUOIO}NOLe NIeH Us sodnon sap IoAOAU .p 

epeurs | JUoUTUIadaI ast v nb uOIsIdgp P| Jo BtUSOg Ud NV.LO.| op xted ve] op usNUuIeW op suonviedo xne 
uonedronied yy ap 1009 FJ ‘OOL *d Op op UONLUOWISNe oUN IOs ‘966[—-SH6I UP SIeT[OP 9p suOTT[T QpZ anbjanb 
® 7661-166] US SIe[[OP op SUOTTIIU /p UOITAUD Pp JULssed ‘sUOTSSTUT Sop d][Te} B] Op 19 SIQUIOU Np UONRIUOWISNe | 
ap o1ins Jed saguue soIgIUJop Soo dYoeTJ US 9IUOUT B JOTOURUTJ INOS aT ‘oI[qnd yUSWINOOp UNONe suep dN 

ou xied ve] op UaNUTeU ap souUaIpeULd suONeIgdo sap ‘IaIoURUTy UOU onb Jo1oURUTy UR} “Teqo[s N09 a] s‘9 


“SJUSUOSULYO op 113k ,s [I NbsIO] ITU91GO & IITOTJJIP Isa snsuasuod 

[21 UF) ‘oTBIDUSS d9TqUIASSY ,[ OP SOIQUIOW CYT Sop JUOUINUASSe | OAR HOP QNO.] ‘soueyodurt soursojas sop 
rorodo moanod snod ‘stoyaynoy, *xted ev] ap UoNuIeUT op suoNeiedo sap gyoRBOTIJO ,| JOJOI[9WIL B JURSIA suONIsOdod 
sinaisnid B a10qeT[Oo & ‘ANO,| op euOjal eT B sOURVOdUTT spuRIs snjd eT sproooe Inb ‘epeueD aT r'9 


“SUOTILSTION Op SPIBLIIL XNILIVs op Issne Inj] asndow Inb “al[nseI yospnq np s}nod 
soy 10X4ed v snqd ua snjd op ouop yuaaras xied ev] op uoTUTeUN Ne sgUT|sap spUOJ so] “snagid stefsp 
Sd] SUBP SUOTIBSTIOD sng] sed yUaINboe ,U SoIQUIOUI Op dIQUIOU UTe}I90 UN ‘epeukD Np smUuONUa TY ® 


‘ANO.] Pp a32Is8 ne uonses 
ap sopoyjsul xnv sogyiodde suonviorewe .p yidap ua “IIssng1 ap souRYd ap nad JUO SUOTSSTU Soy 
‘SOJURSTJJNSUT JUOS SadINOSse sine] onb jo stogiduut yso yepueUt Ine] snbsioj ‘anb speusts o1g\stulJA] oJ e 


‘TIRE Ud Jo otusog Ud ‘epueMY Ne ‘aT[eUIOS Ud suONRIgdO sa] sUOUIEP UO ,[ PUIWOD ‘sagIFUOS JUOS 
Inq Inb sayjauuontppe soyor} Soy JonjoaJo Inod urosaq v a][9 JOP susKOU Sap UONesTULSIC | 19}0P 
B JUSUISATaTIOO sjgid 919 sed UO ,U soIquIoU s}e1q Soy ‘tor ,nbsnf ‘anb sfeusts (NO. 9p UONSeMIp eye 


‘xted e] op UoUIeU op UOTOUOJ 
BS JUDUIDIVITJJO J9IND9X9 Pp UONeSTULZIO J op 9}OVdeod eT JUSITUNT] soJUTeUOS sayURIIOdUIT _p “juUepuaded ‘aTeIpuoU 
g11INDd9s ap syudalqo sas nod yuowINsuI yedrioursd og] epeueD 9] Inod yusInoWIap satus), SUCHEN So] ¢°9 


‘INof j1eJ 1SI,S SOITEPTTU 19 SOTIATO 

sa}IO}Ne So] aNd UONeI9d00d ap 19 UONUIPIOOD ap o]9I UN ‘aNNO UY “19-xned suep onb s}eyq o.NUO UaTq Issne 
xied ve] op UONTsodut,p Ja[OA uN yUa}JOduI0D Inb ‘gytxo[dw09 spueis snyd oun p SUOTSSTU Sop Jond9xe nod QNO.1 
v jodde snjd ua snjd ap ej uo ‘sinof sou oq ‘s}eyq ANUS SONSTUL .P SpJOIOV SI] JO[QNUOS &B NOPINS 11e}STSUOS 
(QNO) Setup suoneN sop UoNestUeSIC .[ op jedroutsd J[OI oJ ‘JuvAvIedny ‘oprlos ausons ey op uy ey sIndap 
gTeoIpes UONRULIOJsURN) OUN Ins & SoTUA) SUOTIeN Sop xed ke] op UoTUTeW ap SUOTeI9do sap oINjeU eT 79 


“xted e] 
ap uaNuUreU op suorsstu sop yiednyd ey] ev sdronsed v epeurd gj ‘sue sap [IJ NY “a[qeIs [PIPUOUW sIpeO UN _p INSLI9IUT | 


ev sked np siLinogs Py Joinsse & 9}sIsuod Inb ‘epeued np siesuens onbryjod e] ap Jyoalqo | op [eNUSd JUSUIg[9 UN Js9 
xred ve] op UUTeU 2] ‘(JOAVIN) [CUONRUIOIUT DOIDUIUIO|D Np jo SoIQSULIID SOMLIFY Sop 1QISTUTW 9] INO i) 


S]ue|[leS S}UIOd — 9 ajideyd 


[D8YDS poul, : ajqosuodsas anaqwrifisaA 
uappvo ‘g papyriy : juiolpv Josguas AnaqvoryfisaA 


epeuey jeuoneusajul 
QOJBWILUOD 38 Sasebuel}O 
souleyy — xied be] ap usl}UIeW 37 


lI 9661 Ie — epeued np [ei9uas Inajeo1JU9A np Woddey 


“g/qenquyuod 9] Inod sfqeuuOsTed INOS UN & ‘9d 19 BULIOJ9I B] Bp synda{go sap ajutane,| B o]quIasua 

Ja]]IVAR) Op Sagssaig{UI sored sap 9}19eded k] ap 39 JIUO]OA BI AP SIOJaInO} eIPUadep SULIOJ9I BI Op Sgdons 

9] JOSLLIOD | ap a[qQISsod 3sa [I 19 asnonNaWIOId Iso DULION B] ‘UONLIIJLIJA B] OP oped 9] SURP S9[I09Ep sowg[qoid 
Sa] Ja[891 Inod sajueyodull semnsow op aIQUIOU UTe199 UN JeUaId JVLIeIZIIIG 9] ‘UONPIIJLIDA EI IP UJ E| V ss 


“uOIsIogid 
S9AB SNUUOD Sed JUAIeIg,U SULIOJ9I B] OP S9SEIULAL $9] 19 S1NOD Sa] onb 9}eISUOD SUOAR SnoU ‘« $]NOD Sop aITTeNNIU » 
B] JNafqo swWOD JUeXIy as US SULIOJ9I B] SLIdaNud kB JOSZIL, NP [LasUOD Np IeLIEI9IN9¢ 9] Is DWQW] P'S 


‘oXed eb] ap uoNeNsturupe,| anb sjo} ‘saxouuod sanbneid ya sauiaysAs sap sureysa9 ap 19 sajsod sop 
UONNLAY,| OP 19 UONBOIJISSE]D L] BP SINOD So] aIINp9I op 19 adEIURARP JaIjsI[duLs op a[qissod sd |] e 


‘auuossad D] ap Si1Odp Sa] ANS auuaIpoUDd 107 ke] dp JUI|NODEp 

Inb ajeLeyes gLred ap saouasixa sa] Jue}edsal Us 1N0} anbrjqnd UOT}OUO]J LE] SUBp Sa}sod sa] sno} 

op uoneNleAg,| aeuLed Inb afqelj 19 Opl[BA JWSUINNsUI UN jOs uEL]d 90 onb snod s9[39I k sojuevodut 

suonsonb sasneiquiou ap juepuadsd aysai [J ‘sinayawiosd s}dadse squreut aodui09 319 uoneJoIgue 
S[QRILIQA GUN ININSUOS JLIEIQINIg a] Jed g10qe|9 saisod sap uonenyeag,p ued nesanou sq] ~—se 


- A9[OI JUSWO|[oNIed onb a19 WeAe,U sULIOJ9I eB] Jed gUasgid IJ9p 9] ‘sue xIs UOMTAUA saude nb NJoUOD sUOAR 
snou “C661 UINf ap UOHeOIFISSe[D Op 9]eI9UDS SULION L] OP UOISIOA B] Bp UONBIIZLIQA aNoU op oyIns eT Vv es 


“a[[Te) Op TOP UN ky] 31eI9,9 ‘a[LOLJIP 93x9}U0D Np nua} a}du0D 

"soisod s9j Jaljissejo B JURAIAS sjouUOIssajoid sodno13-snos 2p 1a sadnois ap aiquiou 9] JUSUa|QeIgpIsUOD aIINpel 19 
odesn Ud JUsWaT}onjOR sued Oy anbyenb sa] Jaoe[duiai nod sajsod sap uonenyeag,p ued [nas un Ja10qe|9,p adreyo 
189.8 JOSPLL NP [IOSUOD Np JVLBIZIIVE 9] ‘sjUaWEsULYS op SUOHEpUeWIWODAL Sap B a}INs JUUOP INOg as 


‘onbiyqnd uonsuoj ve] suep saureumny saoinosse1 Sap uoNsad eb] ap omejnsue auld ve] AUIWOD sagI9PIsUOD JUOS 
saisod sap uonjenyeag,| 19 UONeoIsIssEl Be] ‘aked Re] ap UONLISIUTWIPR | 39 [[eABT op SUOTR[OI So] aNnb stay ‘sautaysAs 
Sonne Pp d9Av sjuBLOdu sual] sing] ap uostes UY “anbi;qnd uoKsu_] ke] op JUSUW9[[9ANOUSI Ne sot] sonbrurgsAs 
sjuswesueyo xnedioutid sap un.| WeMNsuod saisod sap UONEN[LAY,| IP 19 UOTIVIJISSE[D B] AP SULIOJOI bP] ls 


SJUB|IIES SJUIOd — G auyideyy 


4akoQ sanbooe : ajqosuodsa anajywoyia A 
SOpviing DiADW : ajuio[po ajp4gUuas 9D14JDIYf149 A 


enbijqnd uonouoy e| 

SUPP Sa}sod sap uoHenjeng,| ap 

J9 UOHeOISSe/O Be] ap BWUOJOJ C7 
a 


9661 IR — epeury np [e19u9s Ino]eVoTJLIVA Np woddey OL 


‘OIINE OUN ._P NO IIIIULUT SUN, P JULUDJUICUL JUOJ I] SUIOJUT UONPOTJLIDA OP SOOIAIOS 
SOP SNIOUW LI UOITAUD ‘JURPUddaD “JUSWIOPUDI UOS JRINSOUI OU JUIOUT UOTVOIJLIDA Op IOIAJOS UNONe JUsWONbHed 
aonb 938jSUOD SUOTAR SNOU ‘CGH UY “dJQ}STUTU NP JUDWIOUUOTNOUOJ Np UOTWeJOT[9UIe | ve Wodde Ing] ‘aIIp-e-}s9 ,9 

— dUIDJUT UONLOTJLIZA OP SIIAIOS SINI] Ip sOROTJa | JoInsow inod sayenbape sanbruyse} sap 19 sapoyjew sop 
JUDIVSTTNN SdIQ}STUTU So] ap[onbe] suep sINsoUI LT INTLAD SUOAR SNOU ‘UONVITJLIGA 31OU Op dIped o] sued SP 


“SoTTOL9}STUTU 
suoneiado sap sOROTJO | SP 19 DOUSTOTJJO | Op UOTENILAY, | 19 VOUS op 39 9}TWIOJUOD ap suosonb 
Sd] JUSUTUILIOU ‘XNAPT) Sas Op oNpuUd}d | JoIqyInbe XxndUW JOP IUIOIUT UONVOTJLIDA PT ‘dITeJ 90 INOg Lv 


‘Terusjod Uos 3no} SUUOP dUIDJUT UOTROTJLIOA 
ey aonb snod soureuop sone .p suep sogyiodde ang JUDATOp sUONRIOT[aWe sone .p ‘WUepudda ‘ddJOT[aWIe 
}S9,8 SUIOIUT UONVOTJLIDA VT Jed yuouaddoyaagep ud sowiaisAs sop dInjIIaANod eT aonb 93e}sUOd SUOAR SHON 7 


‘QUIOJUI UONPOIJLIDA Be] Op onbyeid ey] Inod sartyqeia soulsou xne sed sinofno} 
JUSULIOJUOD as aU aNenb ‘[Iejep Ud sgUTUTeXS SUOAR SNoU onb S[OLI9}STUTUT SUIDJUT UOTVOTJLIDA Op SddIAJas 3das 
SJ] INS “€G6[ OP UONPOTJLIGA oTJOU sIndop UONeIOT[SWR ,p Nd oJgN3 year A_U [I ‘a[qUIOsUd | sUBp ‘anb 9}e}sUOD 
SUOAR SNOU ‘C66 SINdap dIOI[OUIL 1S9,S DUIOJUT UOTVOTJLIJA Bp SddIAJas sonbyanb ap yusurapud JJ 1S Sr 


“QUIOJUT UONROIFLIDA op onbyijod ey ap d1ANIO Ud SSTU PT OP 9TOKOTIJO ,| 

op uoneNyeAs oun omoutiod Inod sjuesijjns 919 sed UO ,U syJOJJa sao onb suosuad snoy ‘uowexa .p onbytfod 

PR] OP dIPeo J SULP SUIOIUT UOTBOTJLIDA op onbnIjod vy] ap sOVOTJFO ,[ JO[[IOAINS 19 DIANIO Ud ISTUT LI JOSUOART 

inod sonduod 1uos ‘stave os B ‘Inb sayTAT}OR ,P SIQUIOU UIe}IIO UN DUQUI JOSSIL, Np [IasuO_ Np JeUR}VIN0g JT tr 


‘gaLid Ind}Oas np siInotiedns soipeo sap s]fe0 aonb a}uTeNsoI 

snd 189 19-a]]90 Op I[OI NP UOISTA BS ‘OUIOJUT UONLOTJLIZA Op UOIUOJ v] JUOWOTeI9UaS oIndde UOT}SeIIp oIneY PI IS 
auIguI anb 91¥1SUOD SUOAR SNON “UOMSEIIP ayneyY PL] 19 10-a][90 aUS 9\sTxo Inb UONPIAI P| Op d1Ie]NqLy jso suoNesodo 
sop soleuuonses sop soidne ousrajUI UONeOTJLIGA Pe] Wolf yUOp yodsai 9] ‘oINsoU OSE] OUN SURG “UOTeSTURZIO | 

B JUSUIIOVOTJJO ONGINUOD UIOIUT UONLOTJLIVA eT] onb Inod JatUasso 3so UOTOOIIp aney ey op Indde 77 Cp 


“uoTeIOIPOWIe 
Ind] 9p ana us suontsodo.d sap anb tsure ‘sariesturU sap UOT}sas op sonbyeid sap jo s9[o sayfouuonesodo 

10 SOATIVIISTUTLUPR ‘SOIQIOULUT] SOITATIO“ Sop 9ITOBOTIJO | B 19 DOUDTIOIJJO | B JUeND soouvINsse Sop SoISTUTU-snos 
xne ITUINOJ jnod [] ‘TeyWOWIepUOJ UONSES Op [NNO UN jsd DOVdTJJO DUIOJUT UONVOTJLIQA Op SOTAIOS UL) TP 


"S[OLIOISTUTUL SOITATIO’ JO SOUTUUBISOIC Sap dgSTAL 

uonses e] Mueres inod stewel onb ajueyJoduI snq{d isd JdvdTJJO DUIOIUT UOTJVOTJLIQA UN ‘a}X9}UOD 9d suRC “a[O.NU0D 
UN J90I9X9 Pp UONesTuULSIO | INod aiIssad9u LT] Sed JUASINPsI OU SII ‘S[OUUOTJVSTULSIO SO[QUOD Sop NeT] 9] 19 SINJeU 

B] INS JonTJUr WWSANed syUSWIAZULYO $990 Is DUIgJ “SOUUDIPLULD Sd] 19 SUdTIPeURD soy Inod sorjqnd sowwess01d 

SOP IIANIO UD ISTU PT JULIOI[SUIY Ud NO JULUDJUTCUL Ud NO} UONLISIUTUIPe | IP SINOD So] BIINP9I sp oNA US 
SOIOISTUTUT SOT INION.SAI JO VSTTPUOTILI [I : JQuIapIsuOs JUSWTOSULYD op opoliad ouN jIA JUOWIOUIOANOS JT Ur 


SJUBIIIES SJUIOd — p ay!deyD 


UuDOIS “Dd aonsg : ajqvsuodsad AnavoyfisaA 
YJON “H plang : juiolpp josauas AnaywoyfisaA 


SOLUSIUBBIO Sd] 19 S849]SIUIW 
SO] SUBP SUJ9}U! UONEDIJ9A C7 


6 966I IRIN — epeued np [eigugs InayeoTLIZA np yoddey 


‘SoUMUUISOId Sop g}1DVOTJJO | OP asATeUL,] OP 10 

JINSOUI LP] IP SUILUOP I] SULP JUSWUDUIOANOS Np sgoons sop [apy snjd uoNeeseaid oun 3a sayaIOUOD snyd sommsout 
Sop JOANON eIpNey [I ‘sioddes sureysold soy sueg ‘ayUveseINOSUS ode}9 sJQTUIOId SUN ONISUOD Inb JUSWTAUIIANOS 
nv UdWeXd | Ip JUoWOpUdI 9] Ins Woddes uN JUSWOTIeY NB STUINOS & JOSIIL, NP [lasuO_D np yuaprtseid oT 9°¢ 


"S99[QUIOD JUSTOS SaT[9 Nb 9d & JOTI[IDA 9 UDTIXd_pP 

STATOR | IP NVIATU NV SOUNOP] Soy JopA09p ‘UONeNTRAY,p s1QHeUI UD sgyLIOLId sas JUoUIONbHeUIEIsSAs Jo.UOUa 

JOP [I ‘aley 99 INOg ‘oUTBWOP 99 sUeP Sa}LIOLId sas JUDIJQ[Jol UONEN[LAY ,P 9IQHLU US SdIQ\STUTL Sap SoT[aNPrAIpuI 
sgiiolid say anb 20 8 Ja[]IaA a1OOUs }Op JUSUISUIAANOS aT ‘oUgISAs NP UOHeIOQeIP,[ ap sodeig sorgturaId 

XP }S9 Ud UO ‘STOJoINO], USWIEXA _P aIQIeUI Ud sgyLOLId ap 1e}9 We] JUO suLTd sop son xnop sey UOITAUD ‘aguUe 
gruel ey ap sinoo ny ‘alrejaspnq snssoooid 9g] 19 spaLIgistur UONeNyeAD,p sued soy oNUA aT] UN ITQRIa.p 
aiyiqrssod vy ajo ayooidde 9y19- *dUJO]UT UONBOIJLIDA op 19 UOTeNTeAY _p oJeNeUI Us stIduIOD A ‘sourueIZ0Id 

Sop USWeXd Pp oIQILUI Ua sgIIOLId ap stOsse spyATOR .p suvyd sap JUsNoUINOs saraystuTU so] aonb a8rxo sasuadap 
Sop UOT]sad 9p gfeANoual suUIg\SAS aT “S9ID SouTUUeISOId sos Op UONeNTeAY | ap aJUL}SUOD UOISsoIZOId PT JoAndde .p 
12 UONeNyeAY | ap UoNeoytueld ey JasOrpure _p soyqndoosns soinsoul sop giNejSUI & JUSOUIOANOS oT ec 


‘uonednosogid ap sja{ns sap juosNatWap suoNeNeAg sop guyenb eT 39 onpuays /q ‘sero1fauTe 
sed ]UOs as OU SOIQISTUTU Soy SUBP UONLNIeAY,| op UONsEs LI op sjueyIoduT syoodse sureze9 ‘“juepuadoo ‘UOTeNTLAD | 
e Jofoueuyy Indde .p xnevaatu sine] nuayurew UO ‘synod snjd sap sureyza0 g\daoxe ‘sasaisTUTU so] re 


‘oguure Jed sieyjop 

op piel]Iu un ¥ soinotigdns juos sasuadgp sol JUOp souwNeISOId sap gOvOTJa | op UoNeNTeAg,| ev yUeNb sgisoid sap 
Wej UO s]I nb ‘ajduroxe sed ‘g}e1su0d suOAR SNON “sgIo souTtUISOId sap syeI[Nsez soy INS UONPULIOJUI Pp sopuvuap 
xne oipuodas juaanad sjt.nb ganoid quo sazg\sturur sory “afqruodsip sinofno} sed }S9_ U UONPULIOJUT 9919-9) ‘sajydutoo 
oP UONIpper sina][fow oun Jo SUOISIOgp ap ostid vy] nod sourumessosd sap gOVdyJO J INS o[eqo|[s UONeULIOJUT | 

OP JUSWI]eS9 ISIXe JUOWOWIIANOS UOg UL) ‘UOHSeIIp e] & xnatogid smndde sop uIM0J yUoANed sopnid Saq[ai oq 
‘soqJouuonesgdo suonsonb sop ins 39 soureIoid ap soyuesoduros saynad sap ans yuowrayedrourd jusz10d sayya reo 
— OFQISTUTLU NP SOITPUUONSES Sop SUTOSA So] INIJOSSaI oITeJ op JUONUTUOD sUOTeNTeAg seq ‘saqIOLId sas 39 s}919}UI 
Sas Ins oljied apueIs Ua SagpUO]J SUOISIOEp ap Jo[JoI 9] JUOS JONTeAY,p IISTOYD elgisturu un nb suonsonb saq ee 


‘suonenyeag sop aonb sorne suswexa .p d1ped a] sup ssanjdaqJO JUOS asATeur | 19 dINSOUI 

ey onbsioy yuonbrdde ,s tb soursou soy sed astoaid 4 _u uo STRUT “Sopn}d Pp a[NOJ 9UN,p ITUDA JUDANAd gyIORVOTIJa | Op 
asAyeue | }9 aINsoul Be] onb yuowoTeB9 WeuUOdaI K UG ‘souTtUeIOId Sop JUSWAPUSI NP sINsoU ke Op sUTeUIOP 2] sUep 
sourueIsoid ap sareuuoNses sap 9}I[Iqesuodsar ey WeuUODOI JosgiJ, Np [lIasuoD np uswexd.p onbniyjod ayjaanou 
oU/) “SoTIOJIp snjd suonenyeag,p uonesiyegs ey 39 uoNoTURyd ey Npual & NsI[ITW np UoONedyIpour ey] Ze 


“STUINOS JUOS xNeJUSWISUJSANOS soururessoid soy jonbne xnornurw usuexd | op 
uOsTel US WOIO — suonenyead sagjodde juowpunutu0s — souruesZoid sap g119KOTIJ9 ,p Sopnia Sop d1eJ op UIOsaq 
¥] ‘sowmuueisord sap s11ovOJo | JuawWoNbipoyjou JasAyeue Jo JaINsoUr jneJ [I “XneyUSWOUIOANOS sowUeISOId sap 
XNCUTY SIEINSHI S9] INS VONCUOJUL | OP ITU9}GO INOg ‘JoUUONOUOJ op OSE] vs JOIOTTQUIP 19 SUOISID9p Sas JaAndde 

Inod soururesso.1d sap syeqpnsar say Ins UoMeUTIOFUI sINOT[IOU ouN Wey]ey [I.nb nuuosal & JUSWIOUIOANOS 9] Te 


SJUe||IeS S}UIOd — € auyideyg 


L{SAOAIG UDI: ajqosuodsaa AnavoLfisaA 
SOPDAIADG DIAD : aJUIOL[pD a]DAQUAS AILAJDILYIAA 


[e49p9} 
JUBW9UJ9ANOD ne uoenjers7 
Renee ra 


966I IP — epeueg np [Peleus Ino}eoTJLIGA np yoddey g 


‘MUSA | & SJHRTNUINS 
syioddei sya) ap yueAoagid sueyd ap ye A [t,nb snjd uou sed opquias ou [I {yinpold 939 BU O[qRILIGA JNe_NUINS 

yoddes unone 39 ‘aanoeqTas 919 Tor .nbsnf ev sanbrunuruos uoNeULIOsUT | ‘ai[eer U_ “o[qeinp yuoWoddoyoAgp op 
ynoelqo | ap uonestear vy jouttod yo nagid JoLIpusyed ds] UOTAs o[NOJEp as ‘9]UITIO USI }S9 UONRSTSOT[OIE ,p snssado.1d 
a onb Jainsse,s Inod urlosaq JO S[I JUOP UONPUIOJUT,| SINOPIO9p xne JUIssTUINOJ spend syioddvi soTq rg 


‘sjnepnuins syioddes op 

uoneiedgid vy ev jo sjuedronsed say sno} & UONRULIOJUT,p 9OINOS op JURAIOS SaguUOPp op onburq oun “JUSWISUIOANOS 
np s[quiasua | Inod ‘1anyNsuOd op UOTSedI0 || IsTes sed BU UO ‘aNO U_ “sUTOSaq Sno] B s9idepe 919 sInofno} 

sed jU0 .U no syuaT[o xne snudased sinof[no} sed yUOS dU 19-xNdd STRUT ‘UOTRJUSLIO .p SJUOWUNIOP sap 19 SIOT[}e 

sap ‘sajoyid syofoid sap yUOWTUIeJOU ‘sTINO sUTe]IID gJOQeIA & ‘“INd}eITTIOVJ op gy[enb us ‘epeuLD JUoWIOUUOIIAUY 
‘nagid suIUIOD s[NOIgp as snssadoid 9 onb snod agsorjawe og WOP UONPULIOJUT | sp UONsSES BT oz 


‘sastogid ong 
}IOp snssad0i1d np dIANVO Ud ISTUL BT ap aiI[Iqesuodsal BT ‘SOUUISTURSIO So] 19 SOIQISTUTU Soy SUBp onb yUSWOUIBANOS 
Np s[quiasud | SUP 1UR} ‘UOTWRSIZOT[OOY ,.p snssdo0I1d Np sIITIQeinp ev] Jomnsse Inod dryssopesy un ynej [] 7 


‘saassed soouatiadxe sou 

ap suUOde] SAP 1911] JIOOUD INKJ [I sTeUI ‘*Z7HHI SIndap sjIej 919 1UO SaTqeJopIsUOd seIdOId sap ‘STAR aOU Y “oIAINSINOd 
as ossind jI,nb ure sarressaogu juos Inb syusWesuLYd soz sno} JaNjdaIJo .p 19 INOf 99 v sje} sgIsoId SO] JOUTUIRX9 .p 
JUSWIOU 9] 1S9 9 ‘UOTIIsURT) op sported ouN suUep JULUOIUTEUT JSd UOTIeSISO[OIY ,.p Snssad0I1d 9] AUTWIOD C7 


‘adeja .p syioddes op stumnos 

Tu uonoe ,p sueyd op snof ve stu no giedgid sed juowaTeI9UES UO ,U SOUISTURSIO So] 19 SOIQISTUTU Soy “JoJJo UG “nAgid 

919 IIRAR [I SUTUIOD OIANIO Ud sTU 939 sed vu oWIWRISOId 9] “UepUddsD ‘SOUISTURSIO xNe Jo SoJQ}STUTU xne oUTeWOp 
20 SUBP SdITTIGesUOdsal Sop ITeUBISSE [66] US SIOQRIA ‘oTe]JUOWIOUUOITAUS dOUvIES op onbis9yeI}s sIpeo UF) rb b 


‘WWAQUAS ANAIDIYLAA a] ANS 10T B] & So9Odde suoNeoTFIpOU! xe JVOUVISULIOJUOD ‘aIANGO Ud asTW ap sued sap jo 
g[qeinp yuowoddojaagp ap sat8gies sop roredgid ap ‘G66 dIquIod9ep sIndap ‘snud} JUOS SaIg}STUTL Soy STRUT “apod) 
3] UCTS IQUUOTDUOJ op JWOIaNUNUOD souIsTUeSIO sop yednyd ey “xneJUOWOUUOIIAUS UOMO _p sueyd ap uoNeredgid 
eB] pusiduroo mb ‘sayejuowousiaAnos suoneiedo sap « uoNRsIsofOd9 | » Inod uoTeJUSTIO UN }essTUINO; epeued 

Np JUSUIDUIDANOS 9] INOd juawauuosdua,] ap aaUDAIE ap APOD ‘7661 UA ‘XNVIDPYJ SOWISTULSIO Soy 39 SIQ}STUTU 
soy red sonbnead js9 a[eyUSUIOUUOITAUD dOURINS LT ‘(066 [) 1/24 UD] _ 3] SUBP dgOUOUUP 919 B a[[o Nb stndaq ibs 


SJUB|[IES SJULOd — Z auyideyo 


Aaysn]y audoy, : ajqvsuodsad Anajvo1fidaA 
apuojvy ‘y j4aqoy : quiolpp Jpsauas AnaqvdIfisaA 


[eiapa} JUBWAUJEANOH 
ne aje}USsWaUUOJIAUS 
90ueI9b | AP BJANGO Ud ASIW eT 


é 9661 Ie — epeueD np [e19uss Ins}eolyLIEA np woddey 


“epeued ne sorely 
solonplj Sop SUBp SNUS}9p SJHOw Pp SIe][Op op spseT]]IU xnop suIOUT ne .p JosueN | ke WoysueN ne 
sgnbrdde aig v nuadas a] ans jodu,] ap 10J e| OP UOse] eT v JUeNb opmjginbut,p synour XNOLIOS Ope 


“SIBTTOP 9p SUOTT[TU C*/ 
op aurejneUNUIWOS UONeIdepe .p aANeTTUT ouN_p s9}dwOds9 s}e][NSEI Sap Ie[D gouOUD,p onbueul a] e 


: Ns tnb 30 Ans yua}Jod suosrey snou anb suonearasqo so] ST 


‘spurtuexe sed suoAv.u snou anb sjutod sap Ins suorsnouoo sap 191m Inod sTAsas ua.s sed }I0p 
aU IN9}O9I 9] “SojUe}OduIT suorsonb sap jusTeUsIS sargI[NOHIed UONLOIJLI9A Op SUOTRAIASGO Soy Is OWIQI rl 


“STWOU0SY .| ap JULSTJJNS Tonos sues sJoTUap ap sasusdap 
Sg] Ins onb tsure suonestoyne xnv 9yWOJUOS-UOU PI Ins jUSLI0d sagyTeN suoNnsanb Sg] ‘TeJougs UY ‘soreisTUTU 
Sop Soya001 xnv Jo sasuodgp xne yuoyJoddes as inb suoneasasqo sap JusWAT[aMIIgeY oULIOJUAI omideyo a] eT 


‘S2dINOSSal Sop UOHestuUNdo | sp SUOTROTJLIZA SOU ap sINOO Ne NO s9IUE saNne p12 1e1q .P SaI9I90s 

Sop STOLOURUTY S}e19 sop “epeuR_) Np sajdwo_ sap suonesoyNne Xnv gTWOJUOS op SUOTROIJLIQA sou ap 19 sorgroueUuly 
SHONPOTHA SOU 9p SINOS Nv souRSsTeUUOD sLId suOAR snou JOP sasiogid suonsenb sep ins j10ddex suostey 

snou ‘ajideys 20 sued ‘o[quuasue UOs suep JUSWIAUIOANOS Np sayTAnOR xne SdATL[OI SUOTSONb sap Ins yUSIOd 

Inb sapmg.p 39 SUOHROIFLIQA ap npuss a}dur09 un JuouUOp S[I UsIq No ‘sJatfnonJed so1Q\sTUTU Sop suRp ssanqoaqyo 
SUOAL SNOU onb sagIS9}Ul SUONBITJLIIA Sap SIO] SNe] SUOTIEYE}SUOD So] JUSWATTONITGey JUSALIDSp santdeyo sonne gq 
‘yoddes a] suep Jatfnonsed afQ1 un onof « uoNRoTyIOA 9p SUOTJRAIASqO saNY » xnev giovsuod antdeyo oT Fea | 


“SOUNTUTUOS Sop CIQUILY,) B[ 9p UOTUHe | & sagiod ang JWorTeIAgp ‘STAR UOS B ‘INnb soURIIOdUIT_p 
suolsonb say sjtoddei sas suep onour [1,nb terouas INIJVOYLIDA NP IBIxXo9 [D42Uas ANaIDILisAA a] ANS 10T kT Ul 


S}UB]|IES S}UIOd — | auyideyg 


UONBOYUaA 
ap SUOI}EAJaSgO Seujny 
ES 


nee Ane cacao brctny pe 
nid hag Fes ZOliRraUNMTLA Cys a mets 
ie joe tgbeuceernuy gre 

a ipo Stee frente GOsiag pare yee 
ye eer parte OLE 


ar KAS ay jsanace 

eT eae 5, ican » 
be ap tae bga dyn tones om 
gc he ie Peietig aptine 1G ieee 


me? Cidinge | iyo, peoga “hati es foberes 


ant Must i Re Alben Gh Beni 4 > bib qin Wu te 
vil 7 daha Waitin o7 13 wR S Ga baaor: 

MN yay aby ALS Ohi! We cae ucey: b eae ee Gene oD 
din hs parercae Poe Ra Sth ies ‘PONT + ances ie | 
altel tha aaacacalle eMApIGhy wh Conewrgt: eects 


copes AAS iyo. eM - 
am aaa SMteutetty Ge J, ube vot i Latin 
ae a= | Se hiaghueny qe ecaqnhry jt pee Ge paiaenit 
CS sed RAGE ONEGATHTH 
oy, ese CVUISTA — PR OPEL OO INF Ad Anis 510 Ute Gee NONE IITy 


paninye 
hipaa doa FA 7 4 rane Ty Gop). poet venti: TE 


‘ 
wy 


=i, mutase xc Earnie wrt, Cres 
var ifssenet canes 
ae : anne ger cyrratourae oF LAAS iii ate baie a Fe pape pei 
hau pe ageayingncar worasss qos fe siTERTalT ce <4 [Oe M0 Sy | 
| _ Petia nee mn en rete YIN | . 
DSS {irae wman Sar Revehe Saneoneunps yo Seite Abou prepay 


ve 
= 


ah he qe Ansa ssp spaah tanta sz Tyee ane bigeye 
; 1% Wp Pelions oF dene Serer OWE se Wee qu ieauh bh inp ie Fen (a pie ae de £59 aed Ge 


® \ . 


Meise fH 4GG0 
ea ye Caepis Wee 


> id Bigueis 


966] IRIN — BpeuRD np [es9uas Ina}eorfLIaA np yoddey 


‘IIUDA & S9QUUL Sd] SUBP JUOID]USTIO Soy Inb saeite[99 SUOTSIOSp 

So] puoid yUsWIOUIOANOS Ins] onb sdueINsse | OAR JUOSsInd suoIpeuLRD soy onb ulye IUINOF 39 JasTpNN “Mponse1 
JUDATOP SarTeUUOTOUO] Soy aonb UONBUIOJUT,p dIUSS 9] LIIOA {JUSHefeI No JUOUUSTAI s]I.nb asueYyoos op suoT|NyOs 
sap suorssnosadal sa] S[I-JUASsTeUUOD SINOpPIOap SIT {,S}LI[NSEI SIMNIT]IOW Soy JUasTNpold sa{j[a No eT NO spueIs 
snid sa] JUOS SUTOSAG SIT NO RI S9dJIIJJV SO[[I—JUOS SA9ITUT] S9OINOSSOI SIT {SIOBOTJO Issne No} steUI sasna}nod 
SUIOU ISULYIEI OP SUOTINTOS Sep [I-}-9)SIxY {SNpusNe s}e][NsgJ so] JUosTNposd sowueIs0Id say anb 90-48q 


‘souNde] Sop o109U9 aOduION ‘aaNOnpold UOSey ap sgst[NN ang assind aja nb 9310s a|[a} op “uowapIeg 
ne UONBULOJUT,| sp UONeyUasgid eT ‘JUepusdaD ‘suTUeISOId ap UOIs9S sUUO oUN k oITessadguU dryssapRdo] 
np sAnoid yey sed yuo .u Jo at[qejg onbntyod ey ap pres9 | e JUsWeseSus .p onbuew UN aQUOUI UO sanne.q 

‘gssed 9] suep Westey a] aU UO Nb souR\sUOd ap jo dleTIqQLY ,p SNyd SaAv STHNO sdd JUASTTN soteUUONses suTe}IOS 
‘yuoddey np santdeyo soy yonbrputr,| surUIOD ‘sfauasse s[uno sap juasodsip JUSWIOUIAANOS Np saieUUONsSes so] 


‘suoIpeuey sop wednyd ey] op uae J Ins ‘anne 

oun Pp NO UOSeJ UN _.p “OUOP Jo JUSTHOUIOANOS Np UONNOAY | Ins soUSpIOUT oUN JOIN UOT|se3 ap sonbned souuog 
op Janbydde .p g}U0foA PJ Jo UOTeULIOJUT | ap sy[Iqruodsip ey ‘ayUOUNIEd UoNeUIOFUT ouN 39 onbyTTOd op arene 
Uda XIOYD sep pres | & JUOWIAseSsUA UN ‘opr[os JoLIg}sturU dryssepes] uN V0} oUTUeIZOId UN INS sUOTSIOSp So] JOpUoy 
op souvjiodurt | Juaustnos yoddes 99 ap santdeyo sinarsnjg ‘o1qnd np sayuaye saf[aAnou sap 19 soyINpar assoo sus 
S9DINOSSOI IP UOSTLI US JUINJOAY XNEJUSUWOUIOANOS SoMUeISOId Sap 9INJeU P| 19 JUWOUIOANOS Np sinjonIs eT 


91URS ap SUTOS Sa] — epeueD sjuRyequIoD susIoUy Ta 
NUdAAL JJ Ins JOdUIT,.| ap JUSUTOITAY | 2IJUOS oyNT PT — epeuRD nusAdy “I 
sjuenburgp sep uoneidepeal ap soummesisoid sar] — epeued [aUUONdELI09 ddTAIOS ‘OL 


uOHeUSWI[saI 19 UoNOedsut 
: XNBUTUR Sop 9]UBS P| 19 xNeIas9A Sap UONOa}01d ve] — epeueD oNeyUSUTTROISY 19 sININUIsV 6 


g[e.NUIO UONeSTUTUUpR | 
JURIIGe I[QNOWU! | <p UONINSUOS BT — gILINOgS op JUSWOUSTOSUAI Np UsIPeULd dOIAIIS °Q 


o[euoryeu dsusjag — 
BpeULD [PUONPUIDIUT IOIOUIWIOD J9 SalasueNE soneypyY — 
xied ev] op uonureur oy “7 10°°9 


enbrqnd uonouoj ve] suep sojsod sap uonenyeag,| ap 19 UONROYISsETO eT op IUIOJOI BT € 
SOWISTULSIO SO] 10 S3IQ]STUTUT SOT SUBP OUIOJUT UONBOTJLIDA b'T ‘p 

[elapey JUSWISUIOANOS Ne UONeNeAY J XS 

[eJ9paJ JUDUISUJOANOS NL I[eJUDUISUUOIIAUS SOURIS LT OP IANO Ud OSTU PT 76 

: Juoulgsedas 


sst[qnd quos sjauuonrppe sanideyo ozuQ ‘uoneoTytgA ap suONRAIaSqO sorNYW — | oTdey_ np jo sjueyyies syutog 
Sap IAIns }so sodosid—queay 192 966] op Woddes uour ap rew ap SUINJOA 9] IajUaSeId sNOA ap xnaNay sins o¢ 


sodoJd-jueay 


9661 ley — seunwwos 

Sep aiquieyd e] e epeuey np 

jes9uab snayeoyeA np yoddey 
i ee te 


hme 


7 = ad a 
=p. es 


oe 


ee 
anat 


7 Fel pauenpys Sires eee coupe 


| Sennen een pinteieannner) Caer: 
“+ Peninenen ye jo Wap 


any ys gout Sree’. 


Po Senin iyi {Pane 25 SUES (4 > wish 
7 ee : 


TeAaOeien aH TOL i aa a ipantey 


i 
= i 


Died pre wtan te yee 


J _ yv¢ed a cr, hx i Lor =f s 4 wig? e pve 
pau Ay GG Je Retos Ses mea, EY s 
t THEY CAL ESEIUSE? TP AGRO, 
- a a 
Te 6 : 
P : 


gg them Pa uegoe 


F yopye Qoe arayjous2 


‘, Peis Is it Marie Gy ye. | Campo Gee (mes 


as ra eee heey 

i = | 

ah Bs esse aves 
‘Westin coo ft Mdenccron apr ncaa he 


es Pe Calica 


2 


Al 


9661 IRIN — BpeueD np [ei9uas IMoyeoyLaa np yoddey 


We 
61 


LI 


OT 


Sl 


vl 


ik 


OT 


abed 


91ULS OP SUIOS So] — epeuRD sjuRNequIOD susIoUYy 
NUDAII JJ INS JOCUIT,.] BP JUSWIO}IAY J 2IJUOD ONT eT — epeueD nusAcsy 


sjuenburyjap sap 
uoneidepeal op sourumessoid sa] — epeurd [OUUOTIELION 9dIAIOS 


UOTeIUSUII[S9I Jo UOT}OOdsUT : xneUITUL sap 9]URS PT 
19 XNP}9S9A Sap UOT}Da}0Id BT] — epeuRD sITe]UOUTTTROIBY 39 aINTNOUsYy 


9]8U99 UONeISTUTUUpe | JUL IQe o[qnoUIUIT | ap 
UONONSUOS BT — ayLINOS 9p JUSUIOUSTOSUAI NP UdIPeUkd DdIAIOS 


geuoneu osusjaq 
BpeUL, [PUONPUIOJUI SOJOUIWIOZ 19 soJQsUeS someIpy 


xied ev] ap UanUIeUl a7 


anbryqnd uonouoy vy suep 
Sajsod sop uoHeNeAg | Op 19 UONLOTJISSEIO LI Op OUTIOJOI eT 


SOLUSTURSIO SOT 19 SOIQISTUTU SOT SULP OUIOJUT UONROTJLIOA BT 
[eJopaj JUSIISUJOANOS ne UONneNeAg J 


[elgpej JUWIOUIOANOS 
NE B[PJUSWSUUOITAUA JOURS PL] OP IANO Ud IST BT 


UONCIIJLIQA 9p SUONVAIOSgO sonny 


cl 


Ol 


I 


anideyy 


syURTTeS syUIOg 


sodoid—-juvay 


SaJaI}eU Sap ajqe] 


s]uelies S]JUIOd 38 SOdOJd—]ueAYW 


——- 9661 IEIN 


- 


ee 


a TR nipeest 16 WOME (eety ‘5 


ne 7 _ 


fe beaiissbps C2) Ce Lee TI. SP 
i dott gre COUN, ~~ pode) abil Gfi c ‘Does 


“A 


y 
LA) PISS 
: “PV ORF 
ee ap ei iV 

. we 


; - [sear rem; ia RY 


‘SUMNER CLHELe: 
b 


?| CRY? 
joe 
< [nag B 
i One 
y ie 
| 
\ 


4 


if? COyvAyve / 


9661 FeW / IT “VMV.LLO 
VOH ‘sjamesaq stueq “J 


‘epeUueD Np [PI9UIS INDIBOTJLIDA J] 


‘[DAQUAS ANAIDIYIUIA 3] ANS 10 | Op (S)L oydesBesed np suontsodsip 
S2] S9AB STUWLOJUOD US sIQUIRYD ke] & gsodgp ang Wop jenbs] ‘sounuiuIoD sap siquieyD 
Bl & 9661 ep Modder sawed uow juIof-19 amoUIsUe sNoA ap InouUOY,| 1e.¢ 


: SOUNWIWOD Sap sIQUIeYD ke] Ip JUapIselg sjqeiouoy,| V 


VOVNVO NG IWHANSD YNALVOISIYAA VQVNVO dO 1VHY3N3D YHOLIGNV 


0-V6£79-799-0 NASI 
I-1/9661-T V4 “ON JO 
966] BpeURD XNLIUSUTIUIIANOS SIdIAIOG sap jo solfqnd xnearsy Sap SIU, © 


'aJXa] a] 4a8a]]D,p Ing a] supp juawanhiun 
Ja UOIDUIUAISIP AUNIND SUDS ASI]1JN Sa UI]NISPUL aduas a] ‘J4oddp4 Juasasd a] SUD 


‘quawnzop juasasd np ulf 0] B apUuDUIWLOD ap UO a] Jd SA1IJDUL Sap 21qDI. 

Dp} 410A J4oddns asjnv UN ANS 41UaJqgO Sa] NO SJuawWNIOp SA4ND,p 41Ua}qGo ANOd ‘sjsoddns 
suadp Ans ajqiuodsip jsa [1 ‘sjuayjd Sou ap sulosag xnD adpuodas XnNalul ANOg ‘SJUDIIIDS 
sjulog 1a sodosd-quvay un.nb 1suw saspdpys 7] aj4odwuos 966] tut ap jsoddvy aT 


9661 FFIN 


UONEIIUSA Bp SUOHEAIASQO SedINy 
| aayideyd 


S]URIIIES S]UIOd 18 SOdosd—jueAY 


SOUNUWIWIOD Sap sIquivy,yD Pe] B 
epeuey np 

[B19U9S INIJBITJLIOA 
np j10ddey 


966] FFIN 


: UONeOWUEA 9p SUONeAaSgO Saulny 
| euydeyg 


s]ue||Ies S]ulog 38 Sodoid—jueay 


SOUNUIUIOD Sap sIquIvYy,) LT & 

epeuey np 

- [B1aUNS ANVJVILIVA 
np wloddey 


| eid 
S]UB]IIeS S]UIOg 8 SOdold-IL 


apter 2 } Publications 


Report of the 
Auditor General 


of Canada 
to the House of Commons 


Chapter 2 
~The Implementation of 
Fedéral Environmental Stewardship 


May 1996 


Report of the 
Auditor General 
of Canada 


to the House of Commons 


Chapter 2 
The Implementation of 
Federal Environmental Stewardship 


Z.@RA RY 
ZORA» 


MAY 3 11996 
S 

4 S 

sity SSSA / 


May 1996 


This May 1996 Report comprises 12 chapters and a Foreword and Main Points. In 
order to better meet clients’ needs, the Report is available in a variety of formats. 
If you wish to obtain another format or other material, the Table of Contents and 


the order form are found at the end of this chapter. 


© Minister of Public Works and Government Services Canada 1996 
Cat. No. FA1—1996/1—2E 

ISBN 0-662—24467-2 

Copies francgaises aussi disponibles 


Chapter 2 


The Implementation of Federal 
Environmental Stewardship 


The audit work reported in this chapter was conducted in accordance with the 
legislative mandate, policies and practices of the Office of the Auditor General. 
These policies and practices embrace the standards recommended by the Public 
Sector Accounting and Auditing Board (PSAAB) of the Canadian Institute of 
Chartered Accountants. 
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The Implementation of Federal 
Environmental Stewardship 


Assistant Auditor General: Robert R. Lalonde 
Responsible Auditor: Wayne Cluskey 


Main Points 


2.1 Since its announcement in the Green Plan (1990), environmental stewardship has been practised by 
federal departments and agencies. In 1992 the federal Code of Environmental Stewardship provided direction for 
the “greening” of government operations, which includes the preparation of environmental action plans. While 
most agencies will continue to operate under the Code, departments as of December 1995 are required to prepare 
sustainable development strategies and plans for their implementation, following amendments to the Auditor 
General Act. 


2.2 A strategic framework for environmental stewardship, laid out in 1991, assigned responsibilities to 
departments and agencies. However, the program was not implemented as foreseen. This is illustrated by the 
generally poor response rate of departments and agencies in preparing and updating action plans and providing 
progress reports. 


ZS As the greening process is now in a period of transition, it is a good time to review progress to date and 
make any necessary changes to maintain its momentum. In our view, while considerable progress has been made 
since 1992, there are lessons to be learned from past experiences. 


2.4 There is a need for leadership for the greening process both government-wide and within individual 
departments and agencies. Responsibility for the implementation of the process must be clarified. 


2.5 Information management needs to be improved if the process is to remain on track. While Environment 
Canada, as facilitator, has developed a number of tools including pilot projects, workshops and guidance 
documents, these have not always reached clients or been tailored to their needs. Furthermore, the opportunity has 
not been taken to develop a government-wide data bank to be used as an information source for all participants 
and for aggregate reporting. 


2.6 Aggregate reporting provides decision makers with the information necessary to ensure that the greening 
process is focussed, on schedule and moving toward the goal of sustainable development. In fact, reporting has 
been selective to date and no true aggregate report has been produced, nor do there appear to be any plans for such 
aggregate reporting in the future. 
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Introduction 


2.7 Federal environmental 
stewardship is an initiative that was first 


announced in Canada’s Green Plan (1990). 


It is the vehicle being used to demonstrate 
the government’s commitment to 
sustainable development and to ensure 
that all of its operations and procedures 
meet national targets and schedules for 
sustaining the environment. As the 
single-largest business and employer in 
Canada, with an annual purchasing power 
of over nine billion dollars, the federal 
government is in a position to lead by 
example by integrating environmental 
considerations into all of its decisions and 
day-to-day activities. This constitutes the 
“greening” process. 


2.8 The government committed to 
implementing its stewardship initiative 
though the use of the Code of 
Environmental Stewardship released in 
1992. The Code outlined areas to be 
addressed by departments and agencies 
through the development and 
implementation of action plans. The 
reporting of such actions was the means 
by which the federal government 
demonstrated that it is practising what it 
preaches, and by which it was to be held 
publicly accountable for meeting its 
environmental commitments. 


a. In 1992, Environment Canada 
established the Office of Federal 
Environmental Stewardship (OFES) to 
promote and facilitate the implementation 
of the Code. It also signed an 
Environmental Accountability Partnership 
(EAP) agreement with the Treasury Board 
Secretariat (TBS) to improve 
co-ordination between them and, 
primarily, those departments with 
responsibilities for the management of 
lands and buildings. This agreement 
provides a means to build consensus on 
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environmental accountability in 
government operations, through the EAP 
Steering Committee and the various 
technical working groups reporting to it. 


2.10 The stewardship initiative has 
been carried a step further with the 
publication of the Guide to Green 
Government and other documents and the 
amendments to the Auditor General Act 
that came into force on 15 December 
1995. These now require ministers to 
prepare sustainable development strategies 
and plans to implement them, and to table 
the strategies and plans in Parliament by 
December 1997. On behalf of the Auditor 
General, the new Commissioner of the 
Environment and Sustainable 
Development will monitor departmental 
strategies, and report to the House of 
Commons on the extent to which 
departments have met the objectives set 


The federal 
government is ina 
position to lead by 


out in them. The greening process has also 
example. 


become a key element in the 
government’s Pollution Prevention 
Strategy released in 1995. Thus, Canada’s 
journey toward sustainable development 
that began in the 1970s will continue to 
the end of the century and beyond (see 
Exhibit 2.1). These sustainable 
development strategies will replace the 
action plans required under the Code of 
Environmental Stewardship for 
departments and some agencies. 


2.11 Our audit objective was to 
document lessons learned from the results 
of the application of the Code of 
Environmental Stewardship by federal 
departments and agencies to their 
operations, under the guidance of 
Environment Canada’s Office of Federal 
Environmental Stewardship (OFES). We 
looked at Environment Canada’s activities 
as facilitator; at departmental and agency 
response to the requirements of the Code; 
and at the government’s plans for ensuring 
continued progress toward sustainable 
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development. We expect that the results provide important baseline information for 
will assist departments and agencies as the Office of the Auditor General and the 
they deliver on their commitment to new Commissioner of the Environment 
integrate sustainable development into the —_ and Sustainable Development for future 
government’s operations “right across the audit work. Details about the conduct of 
board,” as stated in its June 1995 A Guide the audit are presented at the end of the 

to Green Government. The audit will also chapter. 


Exhibit 2.1 


Canadian Actions on the Road to Sustainable Development 


1971 | Creation of the Department of Environment 


1973 Environmental Assessment and Review Process (EARP) introduced 

1984 EARP Guidelines Order issued by order—in—council 

1986 The Department of National Defence begins environmental baseline studies of its bases 

1988 Canadian Environmental Protection Act (CEPA) proclaimed. Provision under Part IV to make regulations 


that apply to federal works, lands and undertakings for the protection of the environment 


1989 Transport Canada develops environmental management plan for its airports 


1990 Canada’s Green Plan lays out road to sustainable development through the application of federal 
environmental stewardship 


Departments and agencies begin review of environmental implications of existing statutes, policies and 
programs 


1992 Code of Environmental Stewardship introduced and departments and agencies begin preparation of action 
plans required under the Code 


1995 Canadian Environment Assessment Act (CEAA) proclaimed 
Parliamentary review of the Canadian Environmental Protection Act (CEPA) 
The Alternative Fuels Act passed 


“A Guide to Green Government” and “Directions on Greening Government Operations” are released to assist 
departments 


Pollution Prevention Strategy embodies the greening process as a key element for successful implementation 


Auditor General Act amended to create the position of Commissioner of the Environment and Sustainable 


Development, requiring departments to prepare sustainable development strategies and with provision for 
ministers to expand the program 


1996 Expected appointment of the first Commissioner of the Environment and Sustainable Development 


1997 Required tabling of sustainable development strategies by ministers 


1997 & Continued monitoring of sustainable development strategies by the 
beyond Commissioner of the Environment and Sustainable Development and reporting on progress 


2-8 Report of the Auditor General of Canada — May 1996 


Observations 


The Commitment to 
Environmental Stewardship 


Assignment of responsibilities 


2.12 These were clearly laid out in 
1991 together with a long-term strategic 
framework for accomplishing the task. 
The environmental stewardship initiative 
was to be implemented by departments 
and agencies that were to make pledges in 
the form of action plans to carry out the 
intent of the Code of Environmental 
Stewardship. They were to report 
regularly on their accomplishments in 
achieving the goals outlined in their plans. 
Furthermore, they were to implement 
policies and procedures for environmental 
audits. Such audits were to be used to 
measure performance against goals and 
objectives. 


2.13 Environment Canada’s role in the 
implementation of the federal 
environmental stewardship initiative was 
to comprise a series of interrelated 
activities including ongoing strategic 
policy direction and review, program 
development and management, analysis 
and research, liaison and consultation, 
product development and service delivery, 
monitoring and aggregate reporting, and 
program evaluation of OFES. 


2.14 The Office of Federal 
Environmental Stewardship (OFES) was to 
act as the focal point for co-ordination of 
these government-wide efforts to integrate 
environmental concerns into day-to-day 
operations. Its specific responsibilities 
were to include developing an 
interdepartmental environmental network 
to share experiences and information, 
establishing and maintaining a project and 
information data bank, producing 
information/awareness products and 
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services such as guidelines, manuals and 
newsletters and conducting workshops and 


training seminars. Environment Canada 


saw the process as a 
voluntary one. 


Environment Canada’s interpretation of 
its role and responsibilities 


2.15 While the government made a 
commitment to implement the stewardship 
initiative in all departments and agencies, 
Environment Canada saw the process as a 
voluntary one. However, the only 
reference to such a voluntary approach in 
the documents we reviewed was with 
respect to Crown corporations, which 
were excluded from the initiative but were 
to be “encouraged” to adopt the Code in 
their operations. 


2.16 Given this interpretation of its 
mandate, Environment Canada’s approach 
may have sent a mixed message to 
departments and agencies regarding their 
responsibilities under the Code. The 
letters sent to departments and agencies 
used such phrases as “Each department is 
requested” (rather than each department is 
required, as used in the French version) 
“to prepare environmental pledges in the 
form of action plans as stated in the 
Code.” Furthermore, the letters said, 
“There is no specific requirement (for 
agencies) to provide action plans to 
Environment Canada,” although they were 
asked to provide success stories to help 
Environment Canada fulfill its reporting 
responsibilities. In our opinion, this had a 
significant impact both on the way the 
OFES discharged its responsibilities as 
“facilitator” and on the response of 
departments and agencies. For example, 
the production of action plans, progress 
reports and updates was limited and such 
documents were not always forwarded to 
the OFES. 


pea lyf On the positive side, 
Environment Canada sought central 
agency involvement by forming the 
Environmental Accountability Partnership 


Report of the Auditor General of Canada — May 1996 2-9 


The Implementation of Federal Environmental Stewardship 


The partnership 
improved 
co-ordination and 
exchange of 
information. 


There was evidence 
that some action had 
been taken. 


The files contained few 
departmental progress 
reports. 


(EAP) with the Treasury Board Secretariat 
(TBS) in 1992. This partnership, which 
improved co-ordination and the exchange 
of information among participants, played 
and continues to play an important role in 
facilitating the greening process. 


Action Plan Development and 
Implementation Not up to 
Expectations 


2.18 Departments and agencies must 
also bear responsibility for the fact that 
the Code has not been implemented 
completely. Many have been slow in 
preparing action plans and delinquent in 
providing comprehensive progress reports 
showing internal strengths and 
weaknesses. At the time of our audit, the 
OFES had received completed action 
plans for only 16 of 25 departments (later 
reduced from 25 to 19 departments under 
the government reorganization of 
1993-94) and 9 of 78 agencies originally 
contacted by Environment Canada in 
1992. Because the action plan was the 
vehicle that drove the stewardship 
initiative in departments and agencies, we 
felt that an analysis of such plans would 
indicate the areas that were being 
addressed. As the OFES files contained no 
analysis of the contents of these plans, we 
conducted our own. 


Initial analysis revealed several areas 
for improvement 


2.19 Using the OFES guidelines for 
preparing action plans, we analyzed not 
only action plans but also progress reports 
and updated plans. Categories we 
examined included human and financial 
resource allocation, assignment of 
responsibility for environmental 
management, government procurement, 
water conservation, waste reduction, 
energy conservation, land use 
management, human resource 


management, vehicle fleet management, 
environmental audit and “green” office 
practices. Additional categories included 
the use of waste energy for heating and 
other purposes, asbestos management, 
pesticide management, emergency 
response procedures, sewage and septic 
tank management and environmental 
assessment and review procedures. 


2.20 The majority of departments and 
agencies submitting action plans were 
already addressing land use and human 
resource management, and green office 
practices. In addition, some departments 
had begun to address remedial planning; 
others had promoted stewardship 
extensively within their departments, 
including the training of personnel. 


PRIN | In other areas, such as 
government procurement, water 
conservation, waste reduction and energy 
conservation, there was evidence that 
some action was being taken. However, in 
vehicle fleet management little action had 
been taken — in spite of the government’s 
focus on that issue for some time. 
Furthermore, little action had been taken 
by departments and agencies on 
environmental audit, a requirement of the 
stewardship initiative. Such audits can be 
used to measure performance against 
goals and objectives. 


2.22 The files contained few 
departmental progress reports and updated 
action plans. Analysis of the OFES files 
revealed that, of the departments that had 
submitted action plans to the OFES, only 
one had submitted a progress report. None 
had been received from agencies. 
Similarly, only one department and one 
agency had prepared updated plans. As 
with the action plans themselves, no 
analysis of the content of progress reports 
or updated plans could be found in the 
OFES files. 


Report of the Auditor General of Canada — May 1996 


New information obtained from the 
OAG survey questionnaire and other 
sources 


2.23 To supplement and update 
information on the application of the Code 
of Environmental Stewardship, we 
surveyed departments and agencies in 
1995 using a questionnaire we had 
developed and piloted in three major 
departments. We contacted 

19 departments and 61 agencies. All 
received a questionnaire, except for 

16 agencies that were contacted by 
telephone. Completed questionnaires were 
received from all 19 departments and from 
42 of the 45 agencies. 


2.24 We found that 14 of the 

19 departments that completed 
questionnaires had produced action plans. 
Of the 5 remaining, 3 were new 
departments created under the government 
reorganization of 1993-94 and had yet to 
prepare comprehensive plans covering all 
their operations. 


2.25 In agencies the picture was very 
different. Among the 42 completed 
questionnaires we received, only 

13 agencies had what might be considered 
an action plan. We had expected that all 
agencies would have an action plan, even 
if limited to green office practices due to 
the agency’s small size or the nature of its 
operations. 


2.26 Of the 14 departments that had 
produced an action plan, only 7 produced 
a progress report. Thirteen agencies had 
produced an action plan but only 3 had 
produced a progress report. Similarly, of 
the 14 departments that had produced an 
action plan, only 8 subsequently produced 
an updated plan. Of the 13 agencies with 
action plans, only 5 had updated them. 
Given that progress reports and action 
plan updates are necessary inputs for 
reporting on the overall status of the 
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stewardship initiative, we found these 
results somewhat disappointing. 


2.27 Policies and procedures for 
environmental audits have not been 
implemented. When asked in the survey 
questionnaire whether they were 
implementing such policies and 
procedures, 13 departments and 

38 agencies indicated that they were not. 
Furthermore, 7 of the 14 departments with 
action plans had not used environmental 
audits to measure performance against 
goals and objectives. Similarly, 10 of the 
13 agencies that had action plans did not 
verify results through environmental audit. 
However, it should be noted that the 
survey questionnaire did not explore other 


Policies and 
procedures for 

environmental auditing 
have not been 


: implemented. 
options that may have been used to verify Ped 
results. 

2.28 We received additional material 


on federal vehicle fleet management from 
Environment Canada. This included “A 
Manager's Guide to Greening the Fleet” 
(1995) and other material indicating that 
some departments are making progress in 
this area. The Alternative Fuels Act (1995) 
requires all departments, agencies and 
Crown corporations to ensure that by 

1 April 1997 at least 50 percent of 
vehicles in their fleets have engines 
capable of operating on alternative fuels. 


“Government-wide”’ in name but not in 
application 


2.29 The then government’s intention, 
as stated in the Green Plan, was to green 
all of its operations. However, the Code 
does not apply to Crown corporations 
even though they constituted over 

30 percent of government organizations 
contacted by the OFES in 1992. This 
indicates their significance when defining 
the term “government-wide”. Even with 
the changes put in place in 1995, Crown 
corporations remain outside the 
requirements of the greening process. 
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Sufficient information 
has not been received 
by all clients. 


2.30 In fact, the application of the 
stewardship initiative has further narrowed 
in that many agencies were not kept 
informed after initial contact. This may 
explain their generally poor response in 
providing action plans, progress reports 
and updates. Effort by the Office of 
Federal Environmental Stewardship was 
concentrated on departments to maximize 
use of its limited resources. 


Information Management Can Be 
Improved 


2.31 In spite of the small size and 
limited resources of the OFES, it has 
accomplished many of the information 
management tasks assigned to it. Two 
areas requiring further attention are the 
communication of information and the 
development of an information data bank. 


Clients need to be fully informed 


2.32 In 1992, the OFES distributed to 
its client list of 25 departments and 

78 agencies a basic package including a 
copy of the Code, instructions and a 
template for preparing action plans, and 
an explanation of the role of OFES and 
expectations for communication. The 
same package was sent to all clients 
regardless of their size, their activities or 
the environmental impact of their 
operations. The Office of Federal 
Environmental Stewardship designed the 
package to be flexible in its use. 
Departments and agencies were expected 
to adapt it and modify it to meet their 
specific needs. However, the OFES did 
not follow up with all of its clients 
directly to see if this had indeed been 
done and to offer any assistance if it had 
not. 


Da Fs The OFES files show that, 
following the initial contact, information 
was communicated through a network of 
representatives from departments and 


agencies, and through workshops, training 
sessions, newsletters and several 
electronic media including “Environet”, 
‘“Ecofax” and most recently the “Green 
Lane” on Internet. Regardless of these 
efforts, sufficient information has not been 
received by all clients, according to the 
responses by departments and agencies to 
the questionnaire (see Exhibit 2.2). In our 
opinion, better management of 
information will be needed to ensure that 
the greening process is fully implemented 
beyond the closure of the OFES on 

31 March 1997. 


A project and information data bank is 
needed for facilitation and reporting 


2.34 We examined the Office of 
Federal Environmental Stewardship’s files 
to determine the nature and extent of the 
current information available to it. We 
found a great deal of information 
including client lists and correspondence, 
a variety of tools for developing and 
implementing action plans, 
communications networks, and details on 
demonstration projects. However, we 
found that there is no master list of 
eligible entities and no evidence that 
clients had been classified by their 
specific information needs. Files were 
incomplete and out-of-date. This makes it 
difficult to focus effort on departments 
and agencies that require special attention. 
Furthermore, the opportunity has not been 
taken to collect such information for a 
government-wide stewardship data bank 
for use in future decision making. In this 
context, we view the greening data bank 
as constituting a collection of databases 
distributed throughout government 
departments and agencies, including 
Environment Canada. Environment 
Canada would be the focus for requests 
for greening information in that it would 
have a knowledge of and access to the 
government’s greening databases. 


i) 


i) 
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2.35 Government’s stated intention in 
1991 was to establish and maintain a 
project and information data bank. In our 
opinion, knowledge of the size and budget 
of each entity, the environmental impact 
of its operations and up-to-date 
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information on its greening activities is 
necessary information to monitor overall 
progress and facilitate analysis and 
reporting on the initiative 
government-wide. 


Exhibit 2.2 


Sample of Survey Responses 


Responses Relating To Information Management And Co-ordination 


“The department would benefit from information from similar-sized departments and, in particular, from some direction being 
given to departments in leased facilities.” 


“To have more communication between the Office of Federal Environmental Stewardship (OFES), other departments. More 
workshops, guidance.” 


“The most useful function the OFES could serve is as a “window” into Environment Canada. Having one office serving as a 
primary point of contact to identify appropriate officials where technical advice is available, or for sorting out conflicting 
advice, would be extremely helpful.” 


“The co-ordination of partnership efforts, which allows the sharing of expertise and experience with other government 
departments, will continue to be essential.” 


“Better contact, better communications; information may have been distributed before to the chairman or to the secretary 
general but it never got to our directorate.” 


“The OFES should organize an interdepartmental panel to increase awareness with respect to the type of environmental issues 
addressed in the various departments.” 


“Possibly co-ordinate workshops involving other departments.” 


“The first thing that we would require is to receive all the documentation that has been published so far.” 


Specific Instructions And Guidance Requested For Strategy Preparation 


“Further guidance is needed on: 1) Where environmental stewardship fits as an element of sustainable development and 2) 
What other elements make up sustainable development.” 


“We would appreciate receiving samples of other departments’ reports, plans, etc. Also guidelines on how to go ahead with 
environmental management systems, sustainable development strategy, etc.” 


“A government-wide initiative to integrate and recommend a structure to help departments respond to the various judicial and 
political requirements under the Code, the Guide and the Labour Code.” 


“Guidelines for development of strategies and action plans.” 


“More details on content of sustainable development strategy. Outline of what small departments who do not own any 
property are expected to do.” 


“Tt is crucial that central or common service agencies that are mandated to procure goods and to administer buildings 
undertake the responsibility to assist client departments in complying with the aforementioned requirements.” 
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The annual report 
contains only success 
stories. 


Taken together, 
smaller federal 
organizations also 
have a significant 
impact. 


Reporting on the Stewardship 
Initiative Is Incomplete 


2.36 Reporting is an important part of 
accountability, and is the vehicle for 
presenting the results that have been 
achieved. While ministers are individually 
responsible for reporting results of 
greening activities within their own 
departments and agencies, the Office of 
Federal Environmental Stewardship is 
responsible for providing an annual 
aggregate report. In our view, such a 
report should review both successes and 
failures of departments and agencies. The 
OFES informs us that information was 
gathered on unsuccessful approaches and 
was used to develop new tools. However, 
such information was not included in the 
Federal Environmental Stewardship 
Annual Report 1992-1993, or the 
Greening Government Progress Report 
(1903): 


Reporting of progress was not objective 


2.37 The 1992-93 report thus does not 
constitute an objective status report. 
Furthermore, it is of limited use to 
decision makers who provide direction on 
the appropriate use of resources and on 
any changes needed to ensure that the 
stewardship initiative is focussed, on 
schedule, and moving toward the goal of 
sustainable development. 


2.38 The report contains no reference 
to the number of action plans received by 
the Office of Federal Environmental 
Stewardship compared with its original 
contact list, nor does it contain any 
analysis of the contents of those action 
plans. Instead it reports only success 
stories from departments, agencies and 
Crown corporations and makes broad 
statements on the basis of limited 


information. This raises questions about 
its usefulness to clients as a yardstick with 
which to measure their own progress. Also 
open to question is its usefulness to 
Environment Canada’s senior 
management as a briefing document; to 
ministers as a basis for future decisions; 
and to the public as an accountability 
document. 


2.39 
Progress Report (1995) is available to the 
public via the “Green Lane” on Internet. It 
essentially follows the same format as the 
1992-93 report; however, none of the 
limitations noted above are addressed. 


The Greening Government 


The OAG survey provided an update on 
client progress 


2.40 The responses to the OAG survey 
questionnaire provided an update on client 
progress, especially in the agencies on 
which the OFES had little information. 
Environment Canada itself states in the 
Federal Environmental Stewardship 
Annual Report 1992-1993, “Smaller 
federal organizations may not always be 
mentioned, but taken together their 
activities also have a significant impact.” 
Yet of the 42 agencies returning 
questionnaires, 30 indicated that the OFES 
did not maintain contact with them or 
provide documentation specific to their 
needs on preparing action plans and 
reporting results. In many instances they 
had taken action on green office practices, 
which lends credence to the OFES 
statements in the Federal Environmental 
Stewardship Annual Report 1992-1993. 
However, this information was not on file 
at the OFES. Our survey results indicated 
that 17 of the 19 departments that 
responded to the questionnaire have 
maintained contact with the OFES since 
the Code was introduced. 
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There Is Now a “Two-Track” 
Approach to the Greening 
Process 


2.41 With the introduction of the 
Guide to Green Government in May 1995, 
the government has moved further toward 
its goal of sustainable development. In so 
doing, it has in effect created a two-track 
approach to environmental stewardship. 


2.42 Under the amendments to the 
Auditor General Act passed in December 
1995, ministers are required within the 
next two years to prepare sustainable 
development strategies for their 
departments and plans to implement them. 
While these will replace action plans 
prepared under the Code, departments 
could use those plans to assist them in 
developing their strategies. The Guide to 
Green Government requires that 
departments report annually on their 
progress in their Part III of the Estimates. 
Those agencies not covered by the 
amendments to the Auditor General Act 
will continue to prepare and update action 
plans under the Code of Environmental 
Stewardship. 


2.43 The focus is primarily on 
departments, since only three agencies 
have been included under the amended 
Auditor General Act to date. However, it 
should be noted that the Act empowers 
ministers to recommend to the Governor 
in Council other entities under their 
authority that should be covered under the 
Act. Under this provision, selected 
agencies could therefore be required to 
prepare sustainable development strategies 
in the future. 


Departments are now applying the 
Guide to Green Government 


2.44 — Given the high priority that 
ministers have placed on activities 
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covered by the Guide, a series of 
workshops has been conducted across the 
country to bring together federal 
representatives from key departments and 
agencies, including this Office, to discuss 
their roles and obligations under the 
government’s greening process, as well as 
the role of the new Commissioner of the 
Environment and Sustainable 
Development. 


Departments will 
require more specific 
instructions and 


2.45 The responses to our survey advice. 


questionnaire, and information obtained at 
the workshops mentioned above, have 
shown a significant need for specific 
guidance to all departments and agencies 
on how to carry out their greening 
activities. Communications seemed to be 
a major problem given that 7 departments 
(36.8 percent) and 33 agencies 

(78.5 percent) responding to our 
questionnaire indicated that they had not 
distributed the Guide within their 
organizations. 


2.46 As the responses to our 
questionnaire show (see Exhibit 2.2), the 
preparation of sustainable development 
strategies will require that more specific 
instructions and advice be provided to 
departments to enable them to meet the 
December 1997 deadline, while not 
neglecting the needs of the agencies that 


will continue to operate under the Code. The Environmental 


Accountability 
Partnership is 
continuing to provide 
leadership. 


Co-ordination mechanisms are being 
used 


2.47 The Environmental 
Accountability Partnership, established in 
1992, is continuing to provide leadership, 
information and advice to departments 
and agencies, with a goal of improving 
existing co-ordination and alliances 
among them. However, the results of our 
survey questionnaire indicated that active 
participation in the partnership is limited 
to eight departments and only two 
agencies. 


Report of the Auditor General of Canada — May 1996 


ee ee ———— 


The Implementation of Federal Environmental Stewardship 


It is a good time to 
review progress. 


2.48 Recent developments indicate 
that Environment Canada is considering 
the involvement of the Sustainable 
Development Co-ordinating Committee 
and the Interdepartmental Network on 
Sustainable Development Strategies, both 
of which are chaired by Environment 
Canada, to provide strategic policy 
direction to departments and ensure 
co-ordination in the preparation of their 
sustainable development strategies. 


Departments’ development of 
environmental management systems is 
under way 


2.49 No department responding to our 
survey has fully implemented an 
environmental management system 
(EMS). However, there is evidence that 
departments are taking action and giving 
increased attention to environmental 
management. For example, several 
“hands-on” workshops organized by the 
Office of Federal Environmental 
Stewardship have been held using the 
EMS Self-Assessment Guide developed 
with the input of the Office of the Auditor 
General. 


Lessons Learned and 
Recommendations 


2.50 The stewardship initiative under 
the Code of Environmental Stewardship is 
alive but needs attention. That it is 
producing results is substantiated by 
information in the annual reports and by 
the responses to our questionnaire. 
However, not all areas are being addressed 
fully, as noted earlier in this chapter. 


2.51 Since 1990, departments and 
agencies have acquired much experience 
with the difficulties of applying 
environmental stewardship to their 
operations and day-to-day activities. As 
the greening process has now reached the 


stage where departments will be required 
to produce sustainable development 
strategies for presentation to the House of 
Commons and public review, it is a good 
time to review progress to date, and to 
make any necessary changes to keep the 
greening process on schedule and on 
target. Such an assessment will help 
maintain its momentum and focus. 


2.52 Overall, our audit has identified 
several steps that can be taken now to 
improve the implementation of 
environmental stewardship in departments 
and agencies. While we recognize that 
department and agencies are now 
undergoing significant change, we believe 
that improvements can be made without 
major adjustments to the framework 
currently in place, and within the 
established schedule. 


Successful Application of the 
Strategic Framework 


aos In 1991 a sound strategic 
framework for implementing the Code as 
part of the stewardship initiative was put 
in place, including assignment of 
responsibilities to key players. However, 
in the absence of legislation enforcing 
accountability and assigning specific 
responsibilities for outputs, the strategic 
framework was open to interpretation. 
Although the amendments to the Auditor 
General Act have gone a long way to 
address this deficiency, there remain areas 
for further review. 


Strong leadership is the key to success 


2.54 In our opinion, there is a need for 
continuing leadership in the greening 
process both government-wide and within 
individual departments and agencies. 
Somebody needs to accept full 
responsibility for the overall 
implementation of the greening process. 
Such a leader would be aware of 
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day-to-day developments, be in a position 
to provide timely advice to decision 
makers and be able to report 
comprehensively and objectively on the 
status of the initiative government-wide in 
conjunction with the responsibilities of the 
new Commissioner of the Environment 
and Sustainable Development. 


25 In departments and agencies, 
leadership is equally important. In fact, 
we found a strong correlation between the 
quality of action plans and senior 
management’s direct involvement and 
commitment to the establishment and 
operation of an environmental 
management function within the 
organization. 


The Code still applies to most agencies 


2.56 With the introduction of the 
Guide to Green Government in May 1995, 
together with the amendments to the 
Auditor General Act, the government has 
moved the greening process forward. 
However, as we have noted earlier, this 
new regime applies only to departments 
and to selected agencies. Other agencies 
will therefore continue to operate under 
the Code of Environmental Stewardship. 
Co-ordination of these activities will 
become increasingly important when the 
OFES terminates its activities on 

31 March 1997. Crown corporations 
remain formally excluded from the 
greening process. 


PW Environment Canada, working 
with other departments and agencies, 
should clarify those greening activities 
to be carried out under the amendments 
to the Auditor General Act and those to 
be carried out under the Code of 
Environmental Stewardship. 


Department’s response: Agree. 
Environment Canada in co-operation with 
Treasury Board Secretariat, through the 
Environmental Accountability Partnership 


a a A oS SR 
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and other related working groups, will 
continue to provide leadership on the 
greening process, and, in particular, the 
requirements for the Auditor General Act 
and the Code of Environmental 
Stewardship. 


It is the Department's position, however, 
that the Code is a voluntary program 
because of the lack of a legislative 
framework. The AG Act now lays out such 
a framework for promoting departmental 
accountability and assigning specific 
responsibilities for departments and 
selected agencies over time. 
Consideration should be given to 
extending the mandate of the 
Commissioner to include other agencies. 
There is also a need to determine to what 
extent Crown corporations can participate 
in the greening process. 


Somebody needs to 
accept full 
responsibility for the 
greening process. 


Clients Need to Be Fully Informed 
and Involved at All Times 


2.58 In our opinion, it is the role of a 
facilitator to provide good information 
and to establish ongoing dialogue to 
resolve problems and ensure that the 
greening process keeps moving forward. 
This benefits both parties, in that the 
clients are involved and up-to-date and the 
facilitator has the information it needs to 
fulfill its reporting function. In addition to 
the facilitator, departments and agencies 
need to establish such relationships within 
their own organizations if employees are 
to support and contribute to the greening 


Crown corporations 
remain formally 
excluded from the 
process. 


process. 


2.59 Environment Canada should, in 
co-operation with departments and 
agencies, ascertain their information 
needs and take action to address them. 


Department’s response: Agree. 
Environment Canada will continue to 
provide leadership, information and 
advice to departments and agencies 
through the Environmental Accountability 
Partnership. We agree with the Auditor 
General that this partnership will continue 
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There is a still a need 
for aggregate 
reporting. 


to play an important role in facilitating 
the greening process. 


2.60 Senior management of 
departments and agencies should ensure 
that greening information is available to 
all employees and is used at the 
appropriate levels within their 
organizations. 


Department’s response: Agree. 
Environment Canada has, within the 
Department, established an active 
stewardship network of employees 
working toward the development of our 
departmental environmental management 
system. This approach has been shared 
with other departments through the 
Environmental Accountability 
Partnership. 


A Project and Information Data 
Bank Underpins the Process 


2.61 The government’s stated 
intention in 1991 to establish and maintain 
a project and information data bank for 
the successful implementation of the 
stewardship initiative is still valid. As 
noted earlier, the Office of Federal 
Environmental Stewardship has at its 
disposal information that could be used to 
develop such a data bank. While we 
acknowledge that a data bank would not 
necessarily be developed and maintained 
by Environment Canada, as the 
facilitating department it should be aware 
of the existence of greening information in 
other entities and have immediate access 
to it. The areas to be addressed include the 
organization of greening information, the 
location of and access to information held 
by participants, and the provision of ready 
access to the data bank by all interested 
parties. 


2.62 Environment Canada should 
address the need for a government-wide 
greening data bank by placing the 
subject on the agenda of the Sustainable 


Development Co-ordinating Committee 
for early discussion. 


Department’s response: Agree. The role of 
the Commissioner will include monitoring 
and reporting to the House of Commons. 
This monitoring and reporting function 
will likely require the Commissioner to 
develop a data bank of information so as 
to, at a minimum, track the progress of 
departments. We agree to the statement of 
paragraph 2.65 that a number of greening 
issues will require further discussion 
involving the new Commissioner of the 
Environment and Sustainable 
Development. We believe the data bank 
should be among the issues for discussion. 
To ensure the most appropriate and 
efficient use, it is important to determine 
which party will be in the best position to 
develop and maintain the data bank. 


Environment Canada, through its input 
into the Green Lane” INTERNET home 
page, has gathered together and made 
available to the public and other 
government departments and agencies a 
great deal of information on greening in 
general and on Environment Canada 
programs. 


Aggregate Reporting Needs to Be 
Objective and Comprehensive 


2.63 In view of the impact of the 
greening process on federal operations 
across government and its current position 
on the government’s agenda, there is a 
need to provide advice to Environment 
Canada’s senior management, the Minister 
and other government departments on the 
implications of the greening process. In 
1991 the aggregate report was seen to be 
the vehicle for such communication. In 
our opinion, there is still a need for such 
reporting. At this point a comprehensive 
status report on the greening process to 
date would be a useful base from which to 
proceed. Even with the new requirement 
for departments to table sustainable 
development strategies in the House of 
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Commons and to report progress in their 
Part III of the Estimates, there is no 
provision for aggregate reporting on the 
status of the overall greening process. 


Co-ordination Mechanisms Are 
Essential 


2.64 We support the efforts of the 
Sustainable Development Co-ordinating 
Committee and the Environmental 
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Accountability Partnership, as key bodies 
providing leadership, information and 
advice to departments and agencies. 


2.65 This chapter raises a number of 
greening issues that will require further 
discussion both within the federal 
government and between the government 
and the new Commissioner of the 
Environment and Sustainable 
Development. 
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ty About the Audit 


Our audit examined the stewardship initiative and its outputs through which the federal government is 
greening its policies and its day-to-day operations. In previous work, we have looked at specific programs and 
projects to measure the government’s performance against the environmental pledges in the Green Plan and 
its commitment “to lead by doing”. For example, we audited the implementation of the Canadian 
Environmental Protection Act (1991), and the application of the Environmental Assessment and Review 
Process Guidelines Order (1984) to energy megaprojects (1992) and to the Northumberland Strait Crossing 
Project (1995). We have also audited the control and clean-up of freshwater pollution in the Great Lakes 
(1991) and the St. Lawrence and Fraser Rivers (1993), and the management of hazardous wastes (1995). All 
of these audits demonstrated elements of positive progress. Our study on Environmental Management 
Systems (1995) resulted in the publication by Environment Canada of a self-assessment guide to assist 
departments in developing and implementing such systems. 


Objective 


Our objective was to document lessons learned from the results of the application of the Code of 
Environmental Stewardship by federal departments and agencies to their operations under the guidance of 
Environment Canada’s Office of Federal Environmental Stewardship (OFES) (1992 to 1995), thereby to 
provide guidance to government as it delivers on its commitment to integrate sustainable development into its 
operations “right across the board,” as stated in its recently released “A Guide to Green Government (1995). 
The audit will also provide important baseline information for the Office of the Auditor General and the new 
Commissioner of the Environment and Sustainable Development for future audit work. 


Criteria 


We expected that Environment Canada and all departments and agencies would each have clearly defined 
responsibility and accountability for implementing the Code of Environmental Stewardship and for preparing 
sustainable development strategies. We expected that Environment Canada would have an overall strategic 
plan and a structured approach for providing support to its clients that incorporated the activities of the Office 
of Federal Environmental Stewardship (OFES), named as the “facilitator” of the Code. We expected that 
departments and agencies would have developed environmental action plans and strategies, and would have 
implemented them, audited them and reported the results to meet the requirements of the Code. Finally, we 
expected to find that a strategic plan would address the transition to sustainable development strategies and 
the co-ordination of the stewardship initiative following the closing of the OFES on 31 March 1997. 


Scope 


The audit examined three aspects of the stewardship initiative: Environment Canada’s facilitating role, 
responsibilities and accountability (including its Office of Federal Environmental Stewardship) for 
implementing, monitoring and reporting on the initiative; the departmental and agency response to the 


requirements of the Code; and the government’s plans for ensuring that progress toward sustainable 
development continues. 
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Approach 


Our audit focussed on the way the Code of Environmental Stewardship has been applied from its introduction 
in 1992 to the completion of the audit on 31 December 1995. We also reviewed A Guide to Green 
Government and the directives on greening government operations, including the amendments to the Auditor 
General Act. 


The work involved reviewing files, reports and other documentation, and interviews with key Environment 
Canada personnel involved in the stewarship initiative. We analyzed the content of action plans collected by 
the Office of Federal Environmental Stewardship against evaluation criteria we had developed using a 
framework closely modelled on Environment Canada’s draft document, The Evaluation of Environmental 
Action Plans (1995). The Department had used that document to review its own action plans for headquarters 
and regional offices. 


To supplement and update information on the application of the Code, we surveyed departments and agencies 
using a questionnaire we had developed and piloted in three major departments. We contacted 19 departments 
and 61 agencies. In reporting on the stewardship initiative, we used two sets of data. First, data were retrieved 
from the OFES files containing information received from departments and agencies originally contacted by 
the OFES in 1992 (reflecting the government structure of the day). Data obtained through our survey 
questionnaire distributed in October 1995, however, reflect the current government structure. The reader of 
this report should be aware of these differences; for example, since 1992 the number of departments has been 
reduced from 25 to 19. 


Although contacted in 1992 by the OFES, Crown corporations are not covered under the Code or under the 
recent amendments to the Auditor General Act. Consequently, we chose not to include them in our survey 
population, which was based on the definition of “departments” under Schedule I, I.1 and II of the Financial 
Administration Act, updated to reflect current government organization. For the sake of clarity we refer to 
“departments” as those listed under Schedule I and “‘agencies” as those listed under Schedules I.1 and II. 


We analyzed but did not audit the information contained in responses to our survey questionnaire. 
Documentation provided to us to support the responses will be used in future audit work. 


Audit Team 


John Affleck 
Mike Brosseau 
Sassan Ghassabeh 
Geoffrey Robins 


For information, please contact Wayne Cluskey, the responsible auditor. 
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Chapter 3 


Evaluation in the Federal 
Government 


The audit work reported in this chapter was conducted in accordance with the 
legislative mandate, policies and practices of the Office of the Auditor General. 
These policies and practices embrace the standards recommended by the Public 
Sector Accounting and Auditing Board (PSAAB) of the Canadian Institute of 
Chartered Accountants. 
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Evaluation in the Federal 
Government 


Assistant Auditor General: Maria Barrados 
Responsible Auditor: Stan Divorski 


Main Points 


| Government has recognized the need for better information on program results to support decisions and 
to improve the way government works. Information on the “bottom line” of government programs requires 
disciplined measurement and analysis of program effectiveness. The need for studies of program effectiveness — 
commonly referred to as evaluations — is growing as a result of scrutiny being given to government programs. 


3.2 A changed environment has created challenges for planning and conducting evaluations. A new Treasury 
Board review policy recognizes the responsibility of program managers for measuring program performance. The 
policy also recognizes that effectiveness measurement and analysis can come from a variety of studies, but does 
not make clear the standards that apply when such measurement and analysis is conducted as part of reviews other 
than evaluations. 


32 The choice of evaluation topics in departments reflects decisions based largely on departmental interests 
and priorities. Evaluations continue to emphasize the needs of departmental managers — focussing on smaller 
program components and operational matters. These studies can provide valuable support to management. Good 
government also requires overall program effectiveness information for government decision making and broader 
accountability. This is not consistently provided. Departments have shown that they can respond to demands for 
information on the accomplishments of key programs. For example, we found they have made progress in 
assessing the effectiveness of programs that spend more than $1 billion annually. 


3.4 Departments, with the exception of some smaller ones, have maintained their levels of financial support 
for evaluation; however, certain key aspects of the management of evaluation in departments have not improved. 
The scope and quality of evaluations remain matters for concern. 


35 Government has introduced measures with the potential to improve its planning of evaluation and to 
support continued progress in evaluating key programs. A renewed expenditure management system requires 
departments to submit business plans with priorities for reviews of their programs, including evaluations and 
internal audits. This approach has the potential to link departmental evaluation plans to the budgeting process. In 
the first year, about two thirds of the plans identified review priorities. However, the system is in the early stages 
of development. The government still needs to ensure that individual departments’ evaluation priorities reflect 
government’s evaluation priorities. In order to accomplish this, the government needs to state its evaluation 
priorities systematically, identify gaps in review activity and ensure that those gaps are filled. 


3.6 The President of Treasury Board has submitted to Parliament a report on the performance of government 
review that is a promising first step. Future reports need to use more concrete measures and provide a more 
adequate picture of overall government success in measuring and analyzing the effectiveness of programs. 
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Introduction 


Soh What ministers, Parliament and 
the public want to know about any 
government program is what it has cost 
and what results it has achieved. Results 
information can take many forms. For 
example, it may comprise: counts of the 
outputs — the numbers of products or 
services delivered by a program; measures 
of the various dimensions of outputs, such 
as their timeliness and client satisfaction 
with them; or measures of the program’s 
outcomes — the impacts or effects of the 
program on target groups or on Canadians 
in general. It is in the measurement of 
outcomes that evaluation has a unique 
contribution to make. 


3.8 Various social and economic 
indicators are available (for example, 
unemployment rates, crime rates, average 
incomes) to measure success in dealing 
with the problems facing Canada. 
However, establishing the contributions of 
specific programs to changes in these 
indicators is difficult. Assessing what a 
program has accomplished in improving 
the circumstances of Canadians requires 
separate analytic studies. These studies, 
commonly referred to as evaluations, 
involve the measurement and analysis of a 
program’s effectiveness — the extent to 
which it has achieved its objectives. They 
seek to identify the actual contribution 
made by a program. Thus they are an 
essential source of the information 
required for decision making and 
government management, with the 
potential to: 


¢ provide information to support 
decisions about resource allocation; 


¢ help Canadians determine the value 
obtained with their tax dollars; and 
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¢ help public servants manage for 
results and take responsibility for them. 


Jeo Evaluations can provide the type 
of “bottom line” information that is 
needed for government programs. For 
example, as reported in Chapter 6 of our 
May 1995 Report, a study of the Atlantic 
Region Freight Assistance Program, 
initiated by the Policy and Co-ordination 
Group within Transport Canada, found 
that: the subsidy had become less 


important in stimulating regional Evaluations can 


economic growth because costs had provide the type of 
continued to Gene Atlantic eas “bottom line” 

were depending less on markets in central : ; 
Canada; producers in eastern Quebec information that is 
accounted for a large and growing share of —_ needed for 


the subsidy payments made to move goods 
west by truck; and the raw-material 
producing sector represented a rather 
minor part of the overall Atlantic 
economy. In short, the study showed that 
this subsidy program had outlived its 
usefulness. 


government programs. 


3.10 The study presented a clearer 
picture of the program’s contribution than 
could have been obtained from analyzing 
benefits paid to recipients or any single 
indicator of the health of the Atlantic 
economy. The report was widely 
distributed among the public and to 
members of Parliament. In the 1995 
Budget, the government announced its 
intention to terminate the program on 

1 July 1995. 


3.11 An evaluation of parts of the 
training program at Human Resources 
Development Canada is another example 
where valuable information was produced. 
It found that training that is well geared to 
helping workers acquire occupational 
skills in demand in the economy produces 
striking benefits, including improved 
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economic growth and competitiveness for 
Canada. 


Growing need to know if programs are 
effective 


3.12 In 1993, we published three 
chapters on program evaluation. We 
identified interest on the part of Cabinet 
and parliamentary committees for 
evaluation. However, we concluded that 
the system for evaluation was not meeting 
its potential. Following its hearings on our 
audit, the House of Commons Standing 
Committee on Public Accounts reaffirmed 
its interest in program evaluation and its 
belief in the potential benefits that can 
follow from it. 


3.13 Over the last two years, fiscal 
pressures have led the government to 
reconsider many of its programs and to 
recognize the need for better information 
to support decisions. In its response to the 
report of the Public Accounts Committee, 
the government expressed a commitment 
to manage for results and to measure 
results. In his November 1995 report to 
Parliament, Strengthening Government 
Review, the President of the Treasury 
Board confirmed: 


We cannot improve the way 
government works unless we have a 
good idea of how well it is performing 
now. Knowing what works and what 
does not requires credible information 
on performance. 


New developments in evaluation since 
our 1993 audit 


3.14 Evaluations require specialized 
expertise. In many government 
departments and agencies these studies are 
carried out by program evaluators situated 
in units created for this purpose. Studies 
of effectiveness can also be carried out by 
policy analysts and program managers 
elsewhere in departments. 


3.15 At the time of our 1993 audit, 
program evaluation was the key tool for 
the government to examine whether its 
initiatives were succeeding and should be 
continued or whether there were more 
cost-effective alternatives. Accordingly, 
that audit focussed on program evaluation, 
examining how 42 program evaluation 
units in departments across government 
functioned and what they produced, as 
well as the related activities of the 
Evaluation and Audit Branch of the Office 
of the Comptroller General of Canada. We 
recommended improvements to 
departmental management of evaluation 
and increased leadership by the Treasury 
Board Secretariat and the Office of the 
Comptroller General. 


3.16 Since 1993, a number of events 
have affected the conduct of evaluations 
in the government. These have shaped our 
approach to this follow-up audit. Major 
changes have been made to the structure 
of many departments. They have lost, 
gained or redesigned programs or been 
recombined into new departments. The 
restructuring and downsizing that is 
occurring frequently involves changes to 
the structure for evaluation. How 
evaluations are to be conducted in this 
environment presents a challenge for 
many departments. 


Sy Changes have also occurred in 
the central government structure and 
responsibilities for evaluation. The Office 
of the Comptroller General was integrated 
into the Treasury Board Secretariat, and 
the functions of the Evaluation and Audit 
Branch were assumed by the Government 
Review and Quality Services Sector of the 
Secretariat. In May 1994, the Treasury 
Board approved a new review policy as 
well as revised internal audit and 
evaluation policies. The review policy 
establishes the general requirements for 
all types of reviews, including evaluations 
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and internal audits. In particular, it gives 
greater recognition to the responsibility of 
the program manager for measuring 
program performance. The evaluation 
policy sets the specific requirements for 
evaluation. 


3.18 The government announced in 
the 1994 Budget its intention to conduct a 
comprehensive review of all federal 
expenditure programs — the 1994 
Program Review. Key decisions following 
this review were announced in the 
February 1995 Budget. In addition to the 
Program Review, the 1994 Budget also 
announced that 15 policy reviews would 
be undertaken. These reviews were to be 
extensive consultations on a wide range of 
policies and programs with a view to 
improving both economic and fiscal 
performance and government efficiency. 


3.19 The Public Accounts Committee 
considered our 1993 chapters on program 
evaluation at public hearings in June and 
October 1994. In November 1994, the 
Committee reported its conclusions in its 
Sixth Report. Overall, its 
recommendations encouraged stronger 
leadership by the Treasury Board 
Secretariat (a summary of its 
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recommendations pertaining to the 
Treasury Board Secretariat is included in 
Exhibit 3.1). The report also asked this 
Office to examine and report no later than 
the spring of 1996 on the extent to which 
the Treasury Board Secretariat has 
complied with the Committee’s 
recommendations and lived up to 
commitments made during the hearings. 


3.20 In April 1995 the President of the 
Treasury Board responded to the Sixth 

Report of the Public Accounts Committee 
(a summary is provided in Exhibit 3.2). In 


The Public Accounts 


its response, the government provided Committee 
clarifications or alternative approaches to encouraged stronger 
dealing with most of the recommenda- leadership by the 
tions. In November 1995, in response to 

the Public Accounts Committee’s sixth Treasury Board 
recommendation (Exhibit 3.1), the Secretariat. 


government released Strengthening 
Government Review: Annual Report to 
Parliament by the President of the 
Treasury Board (the President’s Report). 


3.21 In February 1995 the government 
announced a renewed expenditure 
management system that would require 
departments to submit business plans to 
implement changes announced in the 
Budget. Among other things, these 
business plans were to provide review 


1. Define in detail the approval process for evaluation plans, spelling out the role and 
responsibilities of each of the parties involved. 


control. 


2. Define criteria for determining when evaluations will come under the Secretariat’s 


Exhibit 3.1 


Summary of Recommendations to 
Treasury Board Secretariat by the 
Public Accounts Committee 


3. Consider additional funding where technical or other difficulties prevent evaluations 
from being carried out within reasonable deadlines. 


4. Provide the committee with a semi-annual progress report on the activities of the 
Government Review and Quality Services Division. 


5. Submit a government-wide evaluation plan to the Committee by October, 1995 and 
every year afterward. 


6. Produce an annual report on departmental evaluation performance, starting on 
31 October 1995. 


7. Require an evaluation plan whenever a new program is approved by Treasury Board. 
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Standing Committee on Public 
Accounts, 30 November 1994 
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Exhibit 3.2 


Summary of Government Response 


to the Recommendations of the 
Public Accounts Committee 


priorities, including priorities for internal 
audit and evaluation, and to make 
collecting information on program 
performance part of departmental 
planning. 


3.22 The 1995 Budget Plan also 
announced a number of measures that 
increase the involvement of other parties 
in the delivery of federal programs. These 
measures complicate the identification of 
roles, responsibilities and accountabilities 
for evaluation. 


Audit scope and objectives 


3.23 Our audit examined measurement 
and analysis of effectiveness in the federal 
government. More specifically, we 
examined the planning, management and 
reporting of evaluations. Evaluations 
included in the audit were identified by 
heads of review in 39 departments and 
agencies as meeting the requirement of 
disciplined measurement and analysis of 
the effectiveness of programs. These 
evaluations were conducted either within 
or outside of departmental evaluation 
units. We examined the business plans for 
the 39 departments and agencies. We also 
examined the related activities in the 


Government Review and Quality Services 
Sector at the Treasury Board Secretariat. 


3.24 Our audit objectives were: 


e to describe the new framework for 
evaluation in government and the changes 
to the structure in place to conduct 
evaluations in departments, and to 
determine the extent to which these 
changes respond to our observations and 
recommendations of 1993; and 


e to assess the government’s progress 
in responding to our 1993 chapters on 
program evaluation and to the 
recommendations of the Public Accounts 
Committee. 


3.25 Further details about the audit are 
presented at the end of the chapter. 


Observations and 
Recommendations 


Some Progress in Evaluation — 
1992-93 to 1994-95 


Various types of studies address 
effectiveness 


3.26 
analysis may be part of different types of 


Effectiveness measurement and 


Treasury Board Secretariat. 


N 


3. Agreed 


4. Agreed 


1. Priorities for review will be examined as part of the annual business planning process by the 


Where Treasury Board initiates reviews, as part of its review of departmental business plans or 
otherwise, it will specify responsibility for their conduct. Typically, Treasury Board Secretariat 
would play a lead role on interdepartmental reviews, where issues have government-wide 
implications, or management issues of particularly high risk. 


5. The government will provide a consolidation of planned and completed key strategic reviews as 
part of an Annual Report of the President of the Treasury Board each fall. Where gaps are noted, 
the Treasury Board will intervene. 


6. The status of departmental review in terms of the government expectations will be included in the 


President’s Report on Review. 
Source: Summary of Government 


Response to the Public Accounts Te 
Committee, provided by Treasury 
Board Secretariat 


The Treasury Board Secretariat will continue to examine all submissions for new programs to 
ensure that these include statements of objectives and commitments to measure performance. 
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studies, including: evaluations conducted 
by evaluation or review units in 
accordance with Treasury Board policies 
and standards; reviews led or mandated by 
senior management or by program 
managers; analysis of information from 
the ongoing monitoring of programs or 
operations; and reviews conducted by or 
at the request of central agencies. 


3.27 Evaluations may deal with 
questions about the overall effectiveness 
of programs and policies (program 
effectiveness) or about the effectiveness of 
specific operations or management 
systems (operational effectiveness). 
Studies that examine operational 
effectiveness are an important element of 
good management. Studies on overall 
program effectiveness are essential in 
assessing the achievements — or the 
“bottom line” — of government programs. 


3.28 In the following paragraphs, we 
examine the evaluations produced in 
departments and, particularly, the balance 
that these strike between measuring 
program effectiveness and operational 
effectiveness. We also examine the degree 
to which two sets of reviews, the 1994 
government-wide Program Review and 
the policy reviews announced in the 1994 
Budget, involved measurement and 
analysis of the effectiveness of key 
government programs. The government 
identified these reviews as having 
assessed critical questions about key 
programs. 


3.29 Priorities for evaluation are 
selected from among the large number of 
programs and policies. Programs may be 
the subject of evaluation for many 
reasons, including, for example, because 
of the resources they expend, because of 
their potential impact on the Canadian 
public or economy or because they 
represent significant aspects of the 
management of government. One set of 
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programs that could be considered a 
priority would be those with annual 
expenditures of over $1 billion. In the last 
part of this section, we examine the extent 
of evaluation of these major programs. 


Evaluations emphasize the needs of 
departmental managers 


3.30 With some exceptions, such as 
the large studies described in Exhibit 3.6 
evaluations in departments and agencies 
generally focus on program components 
and issues that matter more to 
departmental managers than to 
Parliament, the government or the public. 
In 1993 we observed instances where 
evaluations had contributed to the 
modification of government programs or 
to reforms that resulted in savings. 
However, evaluations do not always 
provide needed information on matters of 
broad government policy and broad 
expenditure allocation and are not always 
useful for accountability to Parliament. 
Thus they have served one of their 


> 


Evaluations in 
departments and 
agencies generally 
focus on program 
components and 
issues that matter 
more to departmental 
managers than to 
Parliament, the 
government or the 
public. 


purposes — improvement of operations — 
more than others — accountability and 
support for budget allocation decisions. 


3.31 In four departments, we 
examined all evaluations identified by 
review heads and completed between 

1 April 1992 and 31 March 1995. The 
value of the program components covered 
by these studies ranged from four percent 
of 1994—95 expenditures in one 
department to nearly 60 percent in another 
(see Exhibit 3.3). 


3.32 As in 1993, evaluations continue 
to focus on smaller parts of government 
activities or programs. Government 
programs are organized into activities and 
sub-activities for reporting in the 
Estimates submitted to Parliament and in 
the accompanying Part IIIs. The smallest 
unit reported is typically the sub-activity. 
In 1993 we reported that three quarters of 
evaluations looked at program 
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Exhibit 3.3 


components much smaller than the 
sub-activity. In the four departments we 
examined this time, again three quarters of 
the studies looked at program components 
much smaller than the sub-activity. 


3233 Operational matters continue to 
be a major focus of most evaluations. In 
1993 we reported that more than three 
quarters of evaluations addressed 
operational issues as part of the 
evaluation. In the current audit, between 
62 percent and 100 percent of the studies 
we examined in the four selected 
departments addressed operational issues 
as part of the evaluation (Exhibit 3.4). 


3.34 What expenditures and issues 
each department decides to evaluate is 
influenced by a number of factors, both 
internal and external. In the case of 
Transport Canada, there were demands to 
review significant activities, and many of 
the evaluations that were undertaken 
related to strategic questions such as 


Proportion of Departmental Activities Examined 
for Effectiveness between 1992-93 and 1994-95 


Department 


Proportion of 1994-95 Expenditures 


Transport Canada 
Royal Canadian Mounted Police! 
Revenue Canada 


Health Canada? 


59.2% 
35.1% 
Seo 


4.0% 


1. One key recent review looked at community policing, which has few resources 
attached to it, but which has implications for large activities such as the 
$786.7 million Contract Policing Program. 


2. The Health Insurance Activity, which represented 83 percent of the Department’s 
1994-95 expenditures, was not subject to effectiveness measurement and 
analysis during the three-year period audited, although an evaluation of the 
Department’s monitoring of Health Insurance was under way. Calculation of 
coverage is complicated for Health Canada because of the restructuring of the 
Department — the Welfare component was transferred to Human Resources 
Development Canada during 1993-94. If the calculation is based on 1993-94 
expenditures, coverage becomes 61.9 percent, as evaluations of two large 
programs — Canada Pension Plan Retirement Benefits and Old Age Security — 


were completed. 


whether programs were still relevant and 
whether better alternatives existed. 


3235 The Royal Canadian Mounted 
Police has embarked upon a process of 
organizational renewal. Its evaluations 
have emphasized operational issues. A 
significant proportion of evaluations 
include achievement of objectives or 
identification of alternatives for a variety 
of management, operational or support 
functions. For example, an evaluation of 
the staffing component assessed whether 
specific elements of the staffing process 
meet the objectives and needs of the 
Force, as well as the cost effectiveness of 
specific components of the function. 


3.36 Revenue Canada is responsible 
for administering legislative programs 
related to revenue collection, border 
protection, trade administration and 
income redistribution, for which policy 
responsibilities rest for the most part with 
other government departments. 
Consequently, its evaluations address its 
responsibilities for administering these 
programs and the degree to which 
administration of them contributes to the 
attainment of policy objectives. Revenue 
Canada’s approach to program evaluation 
is to focus on areas of high risk with 
significant strategic value. The 
Department advises us that this is done to 
provide managers with effectiveness 
information to support decision making 
related to the management of change. As a 
result, most studies addressed operational 
effectiveness. Some also examined the 
success of pilot projects or of smaller but 
key components, such as the use of dogs 
to detect illegal drugs. Subsequently, the 
results of such evaluations might be 
included in more comprehensive studies 
of the Department’s operations. For 
example, the evaluation of detector dogs 
was used in the overall assessment of the 
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Department’s border protection role in the 
Canada Drug Strategy. 


3.37 At times, areas chosen for 
evaluation are influenced by external 
demands. For example, Health Canada 
conducted some evaluations as required 
by legislation or Treasury Board 
decisions. In general, Health Canada’s 
evaluations focussed on smaller 
components of programs, such as the 
Department’s responsibilities for 
co-ordinating multidepartmental 
initiatives. Although these co-ordination 
activities do not represent a large 
proportion of its expenditures, these 
evaluations can have important 
implications for the larger programs of 
which they are a part. The Department 
also completed an evaluation of Old Age 
Security and initiated an evaluation of 
Canada Pension Plan Retirement Benefits, 
two large programs that are now the 
responsibility of Human Resources 
Development Canada. It is currently 
examining its monitoring of a third large 
program — Health Insurance. 


3.38 Although each department can 
offer a rationale for the focus and 
approach it takes to evaluations, the result 
overall is a focus on operational matters 
and attention to smaller program 
components. This information can provide 
valuable support to management. 
However, we continue to find weaknesses 
in the supply of information on the 
effectiveness of important programs. In 
value-for-money audits we have reported 
in the past two years, 12 expenditure 
programs did not have available 
information on effectiveness that we 
believed would be helpful to Parliament 
and the public. The expenditures of those 
programs totalled approximately 

$5 billion per year. We also reported that 
such information was not available for 
two tax assistance initiatives totalling 
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approximately $16 billion annually in 
revenue foregone and that evaluations 
were under way in both of these areas. 
The choices made in departments reflect 
largely departmental interests and 
priorities. They do not necessarily result 
in the information on overall program 
effectiveness that is needed to support 
accountability and government decision 
making. 


The 1994 Program Review asked 
departments to examine the rationale 
for their programs 


3.39 In the government’s response to 
the Sixth Report of the Public Accounts 
Committee and in the President’s Report, 
the government-wide Program Review 
announced in the 1994 Budget was 
identified as an examination of significant 
issues of a major portion of government 
expenditures. We examined the extent to 
which this Review had the potential to 
provide measurement and analysis of 
program effectiveness. 


3.40 The purpose of the 1994 Program 
Review was to ensure that the 
government’s diminished resources were 
directed to the highest-priority 
requirements and to those areas where the 
federal government was best placed to 
deliver services. Ministers were asked to 


Exhibit 3.4 


Proportion of Evaluations We Examined 
That Included Operational Effectiveness 


Department Proportion 
Including Operational Effectiveness! 
gi} 
Transport Canada 62% 


Royal Canadian Mounted Police 100% 


Revenue Canada 90% | 


Health Canada 80% 


1. These studies may also have addressed other matters such as the success of the 
program component examined. 
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The 1994 Program 
Review asked 
questions that have 
received little attention 
in program 
evaluations. 


Exhibit 3.5 


1994 Program Review Tests 


review their own programs and activities 
using Six tests: 


e whether the public interest continued 
to be served; 


e the necessity of government 
involvement; 


e whether there was an appropriate 
role for the federal government; 


e scope for partnerships; 
e scope for increased efficiency; and 
e affordability. 


3.41 According to the Minister of 
Finance, the Program Review, which 
encompassed about $52 billion worth of 
spending (excluding the major statutory 
transfers), would lead to program changes 
resulting in a $3.9 billion cut in spending. 


3.42 We compared the six tests used 
for the Program Review exercise with the 
key evaluation questions specified in the 
federal evaluation policy, to assess what 
the exercise could have added to our 
knowledge of program effectiveness. As 
shown in Exhibit 3.5, the tests related 
mostly to the evaluation questions of 
rationale for the program and the cost 


Compared with Evaluation Questions 


1994 Program Review Tests 


Evaluation Questions 


Public Interest* 
Role of Government 
Federalism 
Affordability * 


Rationale or Relevance 


Success in Achieving Objectives 


Public Interest* 
Partnership 


Efficiency 
Affordability* 


Cost Effectiveness of Alternative Design 
and Delivery Approaches 


* Falls within two evaluation questions 
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effectiveness of alternative design and 
delivery approaches. Success in achieving 
objectives, a key element of program 
effectiveness, was not specifically a focus 
of the 1994 Program Review. We did not 
conduct detailed audit work in 
departments on the approaches taken to 
Program Review. 


3.43 Our 1993 audit found that 
evaluations had paid little attention to 
whether government programs were still 
needed, or to the possibility of more 
cost-effective alternatives — issues that 
were the focus of the 1994 Program 
Review. Our examination of all 
evaluations produced over the intervening 
three years in four departments found that 
this was still true in most cases. 


3.44 The 1994 Program Review 
potentially has made an important 
contribution in a large number of 
programs by asking questions that have 
received little attention in program 
evaluations. To the extent that Program 
Review did not measure success in 
meeting program objectives, this gap still 
needs to be dealt with. 


Other reviews of key government 
policies did not always result in 
measurement and analysis of 
effectiveness 


3.45 In its response to the request of 
the Public Accounts Committee for a 
three-year evaluation plan, the 
government pointed to a list of key policy 
reviews announced in the 1994 Budget as 
an indication of the government’s 
priorities for review. These reviews were 
in addition to the government-wide 
Program Review discussed above. The 
President’s Report also pointed to these 15 
policy reviews as priority studies that are 
being pursued. 


3.46 We assessed five of these policy 
reviews with objectives that made the 
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clearest reference to analysis of 
effectiveness. We examined available 
reports to determine the extent to which 
they measured effectiveness. The 
announced focus of the policy reviews 
was on consulting the Canadian 
population to define a vision for the 
future; the five reviews we examined 
relied extensively on consultation. 
Although the majority addressed 
effectiveness issues during the 
consultations, less than half included some 
measurement and analysis of 
effectiveness. We did not assess the 
completeness or quality of that analysis. 


Progress has been made in evaluating 
programs over $1 billion 


3.47 
that can be used to define evaluation 
priorities, including the potential impact 
on the Canadian public or economy or the 
level of expenditures. Programs that spend 
more than $1 billion annually account for 
a large proportion of government 
expenditures. Measuring the effectiveness 
of these programs appears to us to be 
particularly important in view of the 
increased concern about the level of debt 
and deficit, and Parliament’s expressed 
interest in having information on 


There are a number of criteria 


government programs to assist in the 
scrutiny of budgets. 


3.48 In 1993 we reported that only two 
of the 16 programs over $1 billion had 
been evaluated fully during the seven-year 
period ending in 1991-92, and that six had 
had sizeable evaluation coverage. The 
remaining eight programs had received 
limited or no evaluation coverage. 


3.49 As part of the present audit, we 
examined the evaluations of the very large 
programs completed during the three-year 
period (1992-93 through 1994—95) 
following our 1993 audit. We did not 
examine the quality of these studies. We 
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distinguished between studies covering 
operations and studies addressing the 
overall effectiveness of the programs. We 
judged a program to have received 
sizeable coverage if a significant 
proportion of important program 
components had been covered (usually 
ranging from one third to one half or more 
of total program activities) and at least 
some questions on overall program 
impacts had been addressed. We judged 
that a program had received very limited 
or no coverage if less than 10 percent of 
overall program activities were covered. 


3.50 The findings are presented in 
Exhibit 3.6. Where possible, we compared 
coverage during the period 1992-93 to 
1994—95 with our previous estimates of 
coverage over the period 1985-86 to 
1991-92. Because of changes to the 
structure of government programs, we 
were not able to make this comparison in 
all cases. 


3.51 The period covered by our 
present audit was shorter than in 1993 
(three years compared with seven years). 
We took this into consideration in our 
assessment of progress made. A longer 
time frame could produce an additional 
increase in coverage if departments 
maintain the existing momentum in 
program effectiveness measurement 
activity. 


3.52 Exhibit 3.6 shows that five 
programs have received sizeable coverage 
in the three-year period to 1994—95. This 
is a significant improvement since 1993, 
when three of these had received only 
limited or no coverage in the previous 
seven years. 


Seb) Exhibit 3.6 also shows that seven 
other programs received intermediate 
coverage: not negligible but not enough to 
be sizeable. We consider that this is a 
reasonable effort in a three-year period. 


EEE 
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Exhibit 3.6 


Addressing Program Effectiveness 


Studies Completed on Large Main Al 
Programs 1992-93 to 1994-95 ! Estimates Wnter Very 3 
1994-95 Addressing Sizeable 3 mediate > | Limited/No 
Programs _($ billions) | Operations” Coverage Coverage i Coverage :| 
1 
— Management of Public Debt (Finance) 4 41.0 al \~ 
— Income Security including Canada Sie YW a 
Pension Plan (Human Resources Development) 
— Unemployment Insurance 17.1 ad pa 
(Human Resources Development) 
— Health Insurance (Health) > Cash 6.7 al 
Tax Points 8.4 
— Fiscal Transfer Payments — Equalization 8.5 a 
(Finance) > 
— Social Development (Human ell a 
Resources Development) > 
— Education Support (Human Resources 
Development) > Cash 2.6 YW 
| Tax Points 4.1 
— Employment (Human Shall a am 
Resources Development) 
— Air Forces (National Defence) 32 Va el 
— Land Forces (National Defence) 3.0 yw al 
— Indian and Inuit Affairs Program — Program 2S al a 
Funding (Indian and Northern Affairs) 
— Maritime Forces (National Defence) 2.4 Va al 
— Realty Services (Public Works 1.8 a a 
and Government Services) © 
— Transfer Payments to the Territorial 12 ye 
Governments (Indian and Northern Affairs) > 
— Pensions (Veterans Affairs) 1E2 i \~ 
— Operations (Royal Canadian 1.1 (yA al 
| Mounted Police) 
| — Office Facilities (Public Works ie al Va 
and Government Services) ® 
— Central and Common Services 1.0 ed ead 
(Public Works and Government Services) 
— Materiel Support (National Defence) 1.0 ad Va 


1. Large programs that had budgets of $1 billion or more in 1994-95 and 1995-96. Our analysis took into consideration the fact that the period 
covered by our present audit was shorter than in 1993 (three years compared with seven). A longer time frame could produce additional 


increase in coverage. 


2. Studies examined a variety of issues, including the operational effectiveness issues referred to elsewhere in this chapter. 
3. (a) Sizeable Coverage: A significant proportion of important program components have been covered (usually ranging from one third to one 
half or more of total program activities) and at least some questions on overall program impacts have been addressed. 
(b) Intermediate Coverage: Coverage is not insignificant but not sufficient to qualify as sizeable. 
(c) Very Limited/No Coverage: Less than 10 percent coverage of overall program activities — overall program impacts have thus far undergone 


only very limited or no measurement. 


4. The $41 billion represents the interest cost of the public debt. 


The Department of Finance advises us that, for the transfer payments, it did not carry out formal reviews under the Treasury Board review policy 
but conducted internal analysis and intensive discussion with the provinces. 


6. The Department indicates that it established a task force on Real Property in 1994 with objectives to improve client services and devise a 
simpler, more affordable and manageable service delivery structure. 
Note: The National Initiatives program at the Canadian International Development Agency has been subject to many evaluations at the sub-program 
level. This $1.2 billion program has not been included in the exhibit because of the number and diversity of its components and because of 
ongoing phased follow-up work with the Office that involves self-assessment by CIDA. CIDA is also in the process of undertaking a series of 
important performance reviews of its programming priorities, which may provide an improved perspective on overall program impact. 
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3.54 It is important to keep in mind 
that effectiveness may not have been 
measured and analyzed for some 
important components of the programs 
with sizeable or intermediate coverage. 
Continuous attention is needed to ensure 
that the evaluation of all key components 
of these programs is given appropriate 
consideration. 


3.55 Seven programs have received 
very limited or no coverage. Five of these 
are transfer payments to other 
governments. Most of the transfer 
payments had received limited or no 
coverage when we assessed them in 1993 
and therefore have operated for a period 
of at least 10 years without formal 
evaluation. The Department of Finance 
advises us that, for the transfer payments, 
it did not carry out formal reviews under 
the Treasury Board review policy but 
conducted internal analyses and intensive 
discussion with the provinces. The 
Department also notes that these provided 
input to two successive legislative 
renewals of the Equalization program in 
1992 and 1994, and to other major 
changes such as replacing a number of 
these programs with the new Canada 
Health and Social Transfer. It is important 
that ways be found to assess and report on 
the accomplishments of these programs. 


3.56 Finally, we note that 75 percent 
of the programs had coverage of 
operations. 


An | Overall, the current situation 
represents a substantial increase in 
evaluation coverage of this category of 
programs, and therefore of total 
government expenditures. Nonetheless, 
we have identified large programs that 
have been subject to little or no formal 
evaluation over the period audited, some 
of which have received little or no 
attention over the last 10 years. Continued 
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attention needs to be paid to programs 
with intermediate or sizeable coverage. 


Need for Improvement in Systems 
and Procedures 


3.58 In order to maintain the progress 
that has been made in evaluating 
government programs, adequate 
supporting systems and procedures are 
needed. We expected that the key 
elements of these systems and procedures 
would include: 


e integration of evaluation into the 
mainstream of management practices, 
decision-making practices and the culture 
of departments and government as a 
whole; 


e adequate planning to identify and 
address government priorities for the 
measurement and analysis of critical 
questions about the effectiveness of key 
programs; 


e monitoring and reporting on 
evaluation; 


e appropriate structures and 
management processes in departments; 
and 


e establishment and monitoring of 
quality standards for evaluation. 


Steps taken to integrate evaluation into 
the mainstream 


3.59 Evaluation offers the potential to 
improve program operations, identify 
savings and support accountability. To 
meet this potential, evaluation needs to be 
an integral part of the management 
practices and culture of government. 


3.60 In 1993 we observed that 
although some effort had been made to 
link evaluation to the government’s 
planning and budget process, success was 
limited. There was a need to strengthen 
the link. Since then, the government has 
begun to renew its expenditure 


The current situation 
represents a 
substantial increase in 
coverage of large 
programs. 


We have identified 
large programs that 
have been subject to 
little or no evaluation. 
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Performance reporting 
in business plans or to 
Parliament provides 
the opportunity for 
integrating the 
findings of 
evaluations. 


We did not find a 
comprehensive, clear 
and consistent 
statement of 
government evaluation 
priorities. 


management system, the process by which 
it makes decisions on programs and 
spending. We generally support the 
direction being pursued by the 
government in this area. 


3.61 Under the system, three-year 
business plans are to outline how 
departments plan to achieve their program 
priorities within reduced resources. 
Departments are expected to identify 
performance targets and report against 
these targets in future years. They are also 
to list key areas for review through 
evaluations, internal audits or other 
means. Departmental business plans 
therefore provide an opportunity to link 
review and evaluation plans to the 
budgeting process. Performance reporting 
in business plans or to Parliament provides 
the opportunity for integrating the findings 
of evaluations. It also provides a 
mechanism for government, and 
Parliament, to assess progress in 
evaluating programs. 


3.62 The renewed system is still in the 
early stages of development: 1995 was the 
initial year, and the system is still 
evolving. In this initial year, business 
plans were to focus on areas of change 
rather than overall departmental activity, 
which limited the extent to which they 
could contribute to an overview of 
priorities for evaluation. Performance 
reports to Parliament are starting on a 
pilot basis this year. 


3.63 Our analysis of business plans 
found that 65 percent included review 
priorities. It is still too early to judge the 
impact this will have on the planning of 
evaluations in departments and the extent 
to which they will meet government 
priorities. 


3.64 Deputy heads should ensure 
that departmental business plans 


include evaluation findings and identify 
priorities for evaluation. 


No systematic statement of government 
evaluation priorities and no system to 
ensure they are addressed 


3.65 In 1993 we noted that 
Comptroller General staff did not have a 
systematic approach to identifying the 
government’s priority needs for program 
evaluation, and had not ensured that 
individual departmental plans took into 
account what the government needed from 
evaluation. We recommended that the 
Office of the Secretary of Treasury Board 
and Comptroller General develop the 
capacity to identify systematically the 
government’s priorities for evaluation. 


3.66 The government has noted that 
planned reviews were listed in the 1994 
Budget. The 1995 Budget did not continue 
this practice of providing a clear listing, 
although there are scattered references to 
a number of reviews. As noted previously, 
the reviews announced in 1994 did not 
always lead to effectiveness measurement 
and analysis. 


3.67 We did not find a comprehensive, 
clear and consistent statement of 
government evaluation priorities that 
departments can use in developing their 
own plans or that Treasury Board 
Secretariat can use in carrying out its 
government-wide responsibilities. 
Strengthening Government Review states 
that there is a demand for a more 
structured central identification of review 
issues and commits the Treasury Board 
Secretariat to a greater focus on this area. 


3.68 Linking of government 
evaluation priorities to departmental 
plans is unclear. A government-wide 
evaluation plan is one means for ensuring 
that individual departments’ plans address 
government-wide evaluation priorities, 
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and for identifying evaluations that must 
be centrally initiated or led. 


3.69 In 1993, we observed that 
Comptroller General staff did not play a 
leadership role in planning but relied on 
the planning processes in individual 
departments. In its report on program 
evaluation, the Public Accounts 
Committee recommended that the 
Treasury Board Secretariat submit a 
three-year government-wide program 
evaluation plan to the Committee by 
October 1995. 


3.70 In its response to the Public 
Accounts Committee the government did 
not agree with the need for a three-year 
evaluation plan. In its place, it would 
consolidate planned and completed key 
evaluations and reviews of departments as 
part of the Annual Report of the President 
of the Treasury Board. 


Sor Annex C to the President’s 
Report consolidates key completed, 
planned and ongoing reviews reported by 
departments. However, the document 
includes not only evaluations but other 
reviews, without always distinguishing 
their nature. For many of the reviews 
listed, it is not clear whether an analysis of 
the effectiveness of the program was or 
will be part of the review. However, we 
note that other, more detailed 
documentation provided to the Public 
Accounts Committee identifies 
evaluations, audits and reviews. 


3.72 In 1993, we recommended that 
the Office of the Secretary of Treasury 
Board and Comptroller General review 
individual departments’ evaluation plans 
to identify gaps in the coverage of 
identified priorities. The reviews listed in 
Annex C of the President’s Report are not 
compared with a government-wide plan or 
list of priorities. We note that a 
precondition for such a comparison would 
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be a government-wide evaluation plan or 
list of priorities. Neither of these exists at 
this time. 


Se 5) Nevertheless, the President’s 
Report notes that Treasury Board 
Secretariat will provide Treasury Board 
ministers with a report that examines 
review findings and plans in light of 
government priorities. The analysis is to 
identify gaps in the monitoring of 
government-wide issues and an update is 
to be provided in next year’s Report to 
Parliament. 


The reviews in the 
President’s Report are 
not compared with a 
government-wide plan 
or list of priorities. 


3.74 The Treasury Board Secretariat 
should ensure that there is a clear 
statement of specific government 
evaluation priorities. It should monitor 
departmental business plans for gaps 
and omissions with respect to 
government priorities and take steps to 
deal with gaps and omissions. 


Government reporting on review — a 
good first step, but improvements are 
needed in future reports 


33/5 The ability of the government 
and Parliament to assess the adequacy and 
sufficiency of evaluation depends on the 
reporting of progress and results of these 
activities. 


3.76 In 1993 we recommended that 
the Secretary of the Treasury Board ensure 
that information on evaluation activity and 
performance is valid and reliable. The 
Public Accounts Committee 
recommended that, starting on 31 October 
1995, the Secretariat produce an annual 
report on departmental evaluation 
performance. 


3.77 The Treasury Board Secretariat 
notes that the government, in its response 
to the Public Accounts Committee, did not 
agree to all of the recommendations of the 
Committee. The government indicated 
that it would report on: the extent to 
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The first President’s 
Report is a promising 
first step. 


which significant policies, programs and 
operations are reviewed over time; the use 
and impact of findings; areas for 
improvement; and actions taken to correct 
identified weaknesses. A semi-annual 
status report on the activities of the 
Government Review and Quality Services 
Sector of Treasury Board Secretariat is 
also provided to the Public Accounts 
Committee. 


3.78 The first President’s Report — 
a promising first step. The President of 
the Treasury Board tabled the first Annual 
Report on the Performance of Review in 
November 1995. A list of key audits, 
evaluations, reviews and special studies 
was also sent to the Public Accounts 
Committee. We examined the 
performance statements from the 
President’s Report that were within the 
scope of our audit. 


3.79 The President’s Report expresses 
a continuing strong commitment to 
measuring program results and managing 
on the basis of the results achieved. The 
Report presents expectations for 
government review activities, assesses 
some strengths and weaknesses of review 
activities, and sets out some actions for 
the future. This general approach provides 
a promising basis for future reporting. 


3.80 The President’s Report does not 
give an adequate picture of success in 
measuring effectiveness. The ultimate 
objectives of the review and evaluation 
policies are to have timely, relevant and 
evidence-based information on the 
performance of programs, including their 
effectiveness, and to have this information 
used for decision making and 
accountability. Our particular interest was 
to know if these objectives are being met. 


3.81 The recommendations of the 
Public Accounts Committee provide 
examples of areas for which measures 


could be developed. It recommended the 
provision of information on: 


e the percentage of each program that 
has been evaluated; 


e whether the programs have been 
evaluated from the standpoint of relevance 
as well as achievement of objectives and 
cost effectiveness; 


e who used the evaluation studies; 
e the studies’ timeliness; 


e the savings made possible by the 
studies; and 


e the proportion of government 
expenditure covered by the studies. 


3.82 The Committee asked that the 
report be linked to the three-year 
government-wide evaluation plan it had 
also requested. It also specified that other 
forms of review were not a substitute for 
evaluation. 


3.83 The President’s Report does not 
distinguish evaluation from other forms of 
review. Because other forms of review do 
not necessarily address effectiveness 
questions, it is difficult to assess progress 
in effectiveness measurement and 
analysis. For example, estimates of the 
coverage of government expenditures 
differ from those by this Office partly 
because the Report does not separate 
evaluations from other forms of review. 


3.84 The Report provides some 
specific examples of the successes of 
departmental reviews. However, it does 
not provide clear measures of the type 
recommended by the Public Accounts 
Committee, except for reporting on the 
percentage of government activities 
covered by review. Nor does it report 
progress against government priorities for 
evaluation. In general, the Report does not 
focus on the success of evaluation and 
does not provide an adequate picture of 
progress in this area. 
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3.85 The President’s Report presents 
an overly optimistic picture of progress. 
We are encouraged that the Report 
identifies some areas that need 
improvement. Nevertheless, in our 
opinion, the Report provides an overly 
optimistic view of progress for an activity 
that is undergoing major change and 
dealing with many important challenges. 
We found a number of instances where the 
supporting information coming from 
background work undertaken or 
commissioned by the Treasury Board 
Secretariat or the findings of past or 
current audit work of this Office were not 
as positive as the information presented in 
the President’s Report. We also found that 
a number of overall positive statements 
were not well supported. However, the 
Treasury Board Secretariat believes, based 
on the results of its quality assurance 
practices, that the information used to 
prepare the report is quite reliable. 
Nevertheless, the Secretariat 
acknowledges that, in view of the amount 
of information from diverse sources, there 
can be errors in specific items of data and 
different interpretations of findings. 
Secretariat officials note that quality 
assurance practices are being strengthened 
to add credibility to future reports. 


3.86 The Treasury Board should 
ensure that its report to Parliament 
credibly represents the performance of 
review and includes specific measures 
on evaluation. The report should 
include the government’s evaluation 
priorities and progress in addressing 
them. 


Limited improvement in management 
of evaluation in departments 


3.87 Treasury Board evaluation policy 
requires that departments have an 
evaluation capacity. We identified 39 
departments and agencies with active 
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review or evaluation units that performed 
evaluations and were monitored under the 
Treasury Board policy. The management 
of these units is important for ensuring 
progress in measuring the effectiveness of 
government programs. We examined 
changes in the resources and structure of 
evaluation units, and progress in 
improving key aspects of management. 


3.88 Overall expenditures on 
evaluation by evaluation units have 
been maintained since 1991-92. Because 
of the increased organizational integration 
of internal audit and evaluation, many 
review heads had difficulty providing 
reasonable estimates of expenditures on 
evaluation. Department-specific 
comparisons between 1991-92 and 
1994-95 are impossible in a number of 
instances because of the reorganization of 
government departments that started in 
1993. 


The President’s Report 
does not provide an 
adequate picture of 


3.89 Although precise estimates of ; ‘ 
progress in evaluation. 


expenditures on evaluation are not 
possible, evaluation units spent 
approximately $25 million on evaluations 
in 1994-95, down only slightly from 
expenditures in 1991-92. While 
evaluation expenditures have been 
maintained overall, we found that there 
have been sizeable decreases in 
expenditures in a number of smaller 
departments. 


3.90 Evaluation is becoming less 
distinct from other functions. In 1993 we 
observed that, increasingly, program 
evaluation and internal audit were being 
managed jointly. This trend has continued. 
Currently, 56 percent of departments place 
evaluators and auditors in a combined 
unit, compared with 33 percent in 1993. 
The Audit and Evaluation Branch at the 
Royal Canadian Mounted Police is an 
example of totally merged internal audit 
and evaluation functions (Exhibit 3.7). 
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We found limited or no 
improvement in some 
key aspects of 
evaluation 
management. 


Exhibit 3.7 


The Audit and Evaluation 
Branch at the Royal Canadian 
Mounted Police 


3.91 In 1993 we reported that only a 
few studies were carried out jointly by 
internal audit and program evaluation 
units. This time we found that four units 
reported conducting only or mainly joint 
studies that did not distinguish between 
the work of evaluation and that of internal 
audit. Twelve other units sometimes 
conducted joint studies and sometimes 
separate ones. 


3.92 The approaches of internal 
auditors and evaluators and the purposes 
of the end products are different. The two 
functions are guided by different Treasury 
Board policies and standards, and there 
are distinct professional standards for 
internal audit. Regardless of the 
organizational structure, we consider it 
important that the Treasury Board clarify 
the standards that apply to measurement 
and analysis of effectiveness. 


3.93 Limited attention to improving 
key aspects of the management of 
evaluation since our 1993 audit. Review 
heads told us that they had experienced 
difficulties in improving the management 
of evaluation because of the changes 
sweeping through many departments. 
Departments have lost, gained or 
redesigned programs or they have 
recombined into new departments. The 


time taken to finalize departmental 
structures has made evaluation planning 
difficult. Old policies and structures for 
evaluation no longer apply in a number of 
departments and new ones are being 
developed. 


3.94 As a result, we found limited or 
no improvement in some key aspects of 
evaluation management identified in 
1993. At that time, we observed that the 
use of program evaluations was related to 
senior management involvement in 
planning evaluations and in following up 
on evaluation results, and we 
recommended improvements. We also 
noted that few systematic approaches were 
in place to provide a basis for program 
evaluation managers to assess the 
performance of their units. 


3.95 Although the large majority of 
departments still have evaluation plans, 
there are now more departments without 
plans than in 1993 (Exhibit 3.8). In 
addition, only a few departments identify 
priorities for evaluation systematically and 
formally. As mentioned before, 
departmental business plans are supposed 
to include evaluation priorities. This has 
the potential to improve evaluation 
planning. 


evaluations. 


the Commissioner. 


In 1994-95, the Audit and Evaluation Branch used about 36 full-time equivalents and spent 
just over $2.6 million. About 11 of these full-time equivalents and about $0.8 million went 
to evaluation. In 1995, the Branch was reduced to 20 full-time equivalents. 


The Officer in Charge of the Audit and Evaluation Branch reports to the Director, Audit, 
Evaluation & Corporate Services Directorate. This individual reports to the Deputy 
Commissioner, Corporate Management. The Branch does not have auditors and evaluators 
working separately; instead it has review principals and reviewers. The audit and evaluation 
functions are merged; they produce a common planning document and combined audits and 


Between 1992-93 and 1994-95, the Branch produced 11 studies that examined 
effectiveness. Evaluation activities are based on a two-year plan. Plans and evaluation 
studies are approved by the Audit and Evaluation Committee of senior officials, chaired by 


Report of the Auditor General of Canada — May 1996 


3.96 Deputy heads should ensure 
that plans for evaluation are developed 
as part of their business planning. 


3.97 There has been no improvement 
in the availability of mechanisms to 
monitor progress in implementing 
corrective action identified by evaluations. 
Less than two thirds of departments have 
in place a follow-up mechanism. 
However, the business planning process 
has the potential to encourage corrective 
action to address key evaluation findings. 


3.98 There is still no systematic 
process in place in most departments to 
objectively assess and demonstrate the 
value obtained from evaluation. Few 
departments have such mechanisms. Steps 
are being taken to develop frameworks to 
help evaluation managers better assess the 
contributions made by evaluations. An 
interdepartmental committee of review 
officials, with the assistance of the 
Treasury Board Secretariat, is in the final 
stages of developing a framework for 
measuring the performance of program 
evaluation. It is important that evaluation 
managers move quickly to improve the 
performance measurement of their 
evaluation efforts. 


Ongoing concerns about scope and 
quality of evaluations 


5:99 As early as 1983, we found that a 
substantial number of evaluations had 
significant weaknesses in the methods 
used to carry out planned work, 
particularly with regard to measuring 
program effectiveness. In 1993 we 
examined the quality of evaluations in the 
11 programs audited and found it to be 
uneven. We identified a need to improve 
external review of program evaluation. 
We recommended that the government 
ensure that the responsibilities of the 
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Comptroller General for monitoring and 
quality assurance were carried out. 


3.100 The President’s Report 
recognizes that there is some concern 
about the credibility of departmental 
review findings when used for government 
decisions, accountability and public 
reporting. It attributes this concern to 
differences in views on the questions 
addressed, and to the perception that 
departmental review reports may 
sometimes interpret results in a manner 
that could be considered self-serving. 
Background work conducted for the 
President’s Report shows that this concern 
applies especially to evaluation. 


3.101 Our ongoing audit work in 
departments over the past three years 
supports concerns about the scope and 
quality of evaluations. Since 1993, a 
number of our value-for-money audits 
have found that important accountability 
issues were not addressed in evaluations 
or there were problems with the 
methodology used to support conclusions. 
However, we also identified a number of 
evaluations where the quality was 
acceptable. Quality problems occur 
despite the fact that all 39 review heads 
identified for us an extensive range of 
quality control mechanisms. 


3.102 Inhis Report, Strengthening 
Government Review, the President of the 
Treasury Board expresses the intention of 
the Treasury Board Secretariat to address 


Exhibit 3.8 


Proportion of Departments with 
and without Evaluation Plans 


Department 1991-1992 1994-1995 | 
Departments with evaluation plans 41 (98%) 32 (82%) 
Departments without evaluation plans 1 (2%) 7 (18%) 
Total 42 (100%) 39 (100%) | 
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credibility problems by exploring options 
to increase external participation in 
significant review activities. It is 
important that the Treasury Board 
Secretariat move vigorously to improve 
evaluation scope and quality. 


3.103. Lack of clear standards. In 1993 
we observed that the essential components 
of a program evaluation system were in 
place, including the standards to be 
achieved. Since then, Treasury Board has 
revised the policies for evaluation and 
internal audit, and has added a new 
umbrella policy on review. The new 
policy covers reviews that fall outside of 
evaluation and internal audit standards, 
and brings some guidance to bear in these 
areas. 


3.104 The new review policy defines 
evaluation and internal audit; the policies 
for internal audit and evaluation provide 
standards. Standards are provided both for 
the practice of evaluation in departments 
and for individual evaluations. Evaluation 
standards require measurement and 
analysis, including the use of practical, 
cost-effective and objective data 
collection. The standards further clarify 
the issues to be addressed by evaluations. 
However, the review policy does not make 
explicit the standards that apply when 
effectiveness measurement and analysis is 
conducted as part of reviews other than 
evaluations. 


3.105 The quality of studies of 
effectiveness other than evaluations is 
important because of the frequency with 
which departments use them. In 1993 we 
observed that in five of 11 programs 
examined, managers turned to special 
Studies for input to policy development. In 
this audit, we asked review heads to 
identify for us not only completed 
evaluations but also reviews other than 
evaluations that contained a disciplined 
and analytic assessment of the 


effectiveness of programs and operations. 
For the three-year period examined, the 
39 departments identified 347 evaluations 
and 187 other effectiveness reviews. The 
actual number of other reviews is 
probably substantially higher, as not all 
review heads could identify them for us. 


3.106 The Treasury Board Secretariat 
should make clear the quality standards 
to be applied for different types of 

effectiveness measurement and analysis. 


Conclusion 


3.107 The need for good information on 
the effectiveness of programs remains 
strong. At the same time, the government 
continues to commit resources to 
evaluation, particularly in the larger 
departments. We noted in our audit that a 
changed, and still changing, environment 
for planning and conducting evaluations 
creates challenges for many departments. 
Evaluations have had to be planned and 
conducted in an environment dominated 
by reorganizing, restructuring and 
downsizing of government programs. 
Changes to departments and programs 
involve changes to the structure for 
evaluation. At the same time, Treasury 
Board policies governing evaluation and 
related work have changed. They now 
include sources of effectiveness 
information other than the work of 
evaluation units, but do not make clear the 
quality standards that apply to 
effectiveness measurement and analysis 
conducted as part of reviews other than 
evaluations. 


3.108 In this context of change and 
uncertainty, some of the improvements we 
recommended in 1993 to key systems and 
procedures for managing evaluation in 
departments and agencies are not yet in 
place. We still see weaknesses in 
planning, in monitoring action in response 
to the findings of evaluation studies, and 
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in implementing systematic approaches to 
assessing the performance of evaluation 
units. In audits carried out by this Office 
over the past three years, we have 
continued to find problems with the scope 
and quality of evaluations. 


3.109 In 1993 we questioned the 
balance struck between meeting the 
information needs of program managers 
and those of other stakeholders — such as 
the Cabinet and Parliament. In this audit 
we found progress in measuring the 
effectiveness of large programs (those 
with annual expenditures of over 

$1 billion) and in examining government 
programs and policies from the 
perspective of their continuing relevance 
and the cost effectiveness of alternatives. 


3.110 Although departments have 
shown that they can respond to demands 


for information on the accomplishments of 


key programs, most evaluation units in the 
departments we examined have continued 
to focus on smaller program components 
and the operational needs of program 
managers. This is an important and 
necessary role but, by itself, would leave a 
gap in effectiveness information needed 
for accountability and strategic decision 
making. 


3.111 In 1993 we also identified the 
need to improve government-wide 
systems and procedures for evaluation — 
particularly in relation to the 
government-wide planning of evaluation 
and the leadership role of the Treasury 
Board Secretariat. The Public Accounts 
Committee subsequently made a number 
of related recommendations specifically 
addressed to the Treasury Board 
Secretariat. We note that measures 
currently being implemented under the 
government’s renewed expenditure 
management system have the potential to 
respond to many of our recommendations 
and those of the Public Accounts 
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Committee. However, we found that two 
key elements are still missing —a 
systematic identification and 
communication of the government’s 
evaluation priorities, and a clear 
mechanism for assessing departmental 
plans against those priorities. 


3.112 The Public Accounts Committee 
recommended that the approval process 
for evaluation plans be spelled out in 
detail. Responsibilities for evaluation 
planning are spelled out in the Treasury 
Board review policy and in the new 
business planning guidelines. Criteria for 
determining when evaluations will come 
under the Secretariat’s control are not set 
out in advance. Decisions on Treasury 
Board Secretariat involvement and on 
what special funding is required are made 
on a case-by-case basis. As requested by 
the Public Accounts Committee, the 
activities of the Government Review and 
Quality Services Sector are reflected in 
the semi-annual status report given to the 
Committee. The Treasury Board 
Secretariat is pursuing the examination of 
submissions to the Board with a view to 
requiring an evaluation framework for 
new programs. 


3.113. The President’s Report, the 
annual report on the performance of the 
review function in government, is a step in 
the right direction. The general approach 
of setting expectations, assessing strengths 
and weaknesses and establishing an action 
plan is promising. However, to respond 
more fully to the Public Accounts 
Committee’s request for a report on the 
performance of the evaluation function, 
future reports need to provide a more 
adequate picture of overall success in 
evaluating government programs. 


Treasury Board Secretariat’s comments: 
This chapter confirms the considerable 
progress of the government in reviewing 
its programs. The Treasury Board 
Secretariat is dedicated to helping 
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departments deliver cost-effective 
programs for Canadians while sustaining 
a management culture that is fact-based, 
results-oriented, open and accountable. 
Effective review is an important part of 
this. We welcome the chapter as 
supporting our actions in the pursuit of 
this objective. 


Program Review and the first President's 
Report to Parliament on review — 
Strengthening Government Review — 
were major Steps in putting the focus on 
results in serving Canadians. For the first 
time, departments examined their 
programs from the perspective of the 
Program Review tests. They described the 
management of their review activities in 
terms of current expectations, and made 


public the list of both completed and 
planned key reviews. These developments 
are most significant. 


The President, in his Report to Parliament 
on review, recognized that more needs to 
be done to strengthen the government's 
review activities and published an action 
plan. This chapter is supportive of the 
government's commitments. The Treasury 
Board Secretariat ts actively pursuing 
each initiative in the government's action 
plan. 


We welcome continued monitoring of the 
government Ss progress in measuring 
results and performance, as public 
attention and accountability are important 
to effective performance. 
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ty About the Audit 


Scope 


Our audit examined measurement and analysis of effectiveness in the federal government. More specifically, 
we examined the planning, management and reporting of evaluations. Evaluations included in the audit were 
identified by heads of review in 39 departments and agencies as meeting the requirement of disciplined 
measurement and analysis of the effectiveness of programs. These evaluations were conducted either within 
or outside of departmental evaluation units. We examined the business plans for the 39 departments and 
agencies. We also examined the related activities in the Government Review and Quality Services Division at 
the Treasury Board Secretariat. 


Objectives 


Our audit objectives were: 


e to describe the new framework for evaluation in government and the changes to the structure in place to 
conduct evaluations in departments, and to determine the extent to which these changes respond to our 
observations and recommendations of 1993; and 


e to assess the government’s progress in responding to our 1993 chapters on program evaluation and to the 
recommendations of the Public Accounts Committee. 


Approach 


The criteria used for the audit were the same as in 1993. In conducting our examination, we administered a 
questionnaire and interviewed heads of review about the structure, planning, reporting, management and 
resourcing of evaluations in 39 departments and agencies. We also examined their business plans. In four 
departments, we assessed the scope of all evaluations completed between | April 1992 and 31 March 1995, as 
identified by review heads. We also conducted focus groups with heads of review to obtain their views on the 
directions evaluation is taking in the federal government and the factors shaping these directions. 


We met with Treasury Board Secretariat officials and assessed supporting background information for 
commitments made in the government’s response to the Public Accounts Committee and for information 
contained in the President’s Report. 


Audit Team 


Henno Moenting 
Jim Blain 

Sylvie Cantin 

Janet Hatt 

Brenton MacDonald 
Christine Pépin 
Sylvie Soucy 

Glenn Wheeler 


For information, please contact Stan Divorski, the responsible auditor. 
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Chapter 4 


Internal Audit in Departments 
and Agencies 


The audit work reported in this chapter was conducted in accordance with the 

legislative mandate, policies and practices of the Office of the Auditor General. 

These policies and practices embrace the standards recommended by the Public : 

Sector Accounting and Auditing Board ( PSAAB ) of the Canadian Institute S 
Chartered Accountants. : 
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Assistant Auditor General: David H. Roth 
Responsible Auditor: Bruce C. Sloan 


Main Points 


4.1 The government is going through a period of tremendous change, in which departments are being 
streamlined and restructured in an effort to reduce the administrative cost of government while maintaining or 
improving the delivery of government programs to Canadians. Although these changes may influence the nature 
and location of organizational controls, they have not lessened the need for the organization to be in control. 
Within this environment, the need for effective internal audit has never been more important to ensure that 
departmental programs and operations are properly managed. 


4.2 An effective internal audit unit is a fundamental management tool. It can provide deputy ministers with 
assurances about the efficiency and effectiveness of key financial, administrative and operational activities and 
departmental management practices, along with suggestions for improvement. 


4.3 The support of senior management is essential if internal audit is to be an effective, contributing part of 
an organization. To a great extent, the relationship between internal audit and senior management determines the 
respect internal audit receives from operating managers. We found that, while senior management expressed 
general support for the internal audit function, its perception of the role of internal audit was narrower than that of 
senior management in the private sector. 


4.4 The Treasury Board Secretariat carries out a number of activities that it believes are designed to further 
the implementation and monitor the effectiveness of the internal audit policy, one element of the Review Policy. 
In our view, these efforts have not been sufficient to provide an assessment of the effectiveness of the 
government’s internal audit policy. 


4.5 While some internal audit units have improved their performance since 1993, we found that, overall, 
there has been little improvement since our 1993 audit. Of the seven departmental internal audit units we 
reviewed in detail, four do not meet established standards for the practice of internal audit. 


4.6 We noted that internal audit coverage of computer systems under development has improved. However, 
further improvement is still needed in other areas if internal audit is to fulfill its potential. 


4.7 To fulfill that potential, internal audit must improve the balance of the scope of its work, including both 
compliance and control issues and the assessment of the efficiency and effectiveness of departmental operations. 


4.8 As part of our audit, we assessed the extent to which departments were using appropriate methods and 
techniques for measuring the effectiveness of their internal audit units — that is, their contribution to improving 
the way a department operates. We found that in 1993 virtually no audit units were measuring their own 
performance. However, approximately half of the audit units are now doing so in some way. 
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Introduction 


4.9 Internal audit has two primary 
tasks. The first is to review and 
independently assess the management 
practices associated with an organization’s 
key financial, administrative and 
operational activities. The second is to 
recommend to management where 
improvements can be made. 


4.10 Having a well-functioning, 
effective internal audit organization in his 
or her department should be of utmost 
importance to every deputy minister. It is 
important because internal audit can be 
extremely useful in helping a deputy head 
remain accountable to the minister and, in 
turn, strengthening the minister’s 
accountability to Parliament for using 
resources efficiently and effectively. 


4.11 Many private sector companies 
that have gone through the same degree of 
change that the government is currently 
experiencing see internal audit as a critical 
element of their management and control 
framework. They have found that internal 
audit contributes significantly to both 
improving the way the organization 
operates and helping management to 
achieve its organizational objectives. 


4.12 In our view, this is the role that 
internal audit can and should be playing 
— but is not yet playing — in the federal 
government. In fulfilling this role, internal 
audit’s first responsibility is to provide 
senior management with sound 
information and advice about the 
adequacy of the organization’s systems of 
internal control and on how well they are 
performing. By providing this type of 
information to management, internal audit 
is helping management to achieve its 
business objectives and fulfil its operating 
responsibilities. In so doing, a strong 
internal audit function can help the 
government to achieve its twin objectives 
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of creating a less costly, more innovative 
and service-oriented public service while 
maintaining probity and prudence and 
strengthening accountability. 


4.13 Within the federal government, a 
number of organizations/units have 
well-defined responsibilities for the 
effective implementation of the internal 
audit policy. Exhibit 4.1 lists these 
organizations/units and their respective 
responsibilities. 


4.14 In carrying out these 
responsibilities, the government spent 
about $48 million in 1994—95, and 
employed about 590 internal auditors in 
39 departments and agencies. This reflects 
a decrease from 1993. In that year, 
resource expenditures were $56.6 million, 
and 700 internal auditors were employed. 


Internal audit can be 
extremely useful in 
helping a deputy head 
remain accountable to 
the minister and, in 
turn, strengthening the 


minister’s 
The Current Environment accountability to 
4.15 The environment within which Parliament. 


the government is operating has continued 
to evolve since 1993, when we last 
conducted a comprehensive review of 
internal audit. Through its budget 
statements, the government has introduced 
a number of far-reaching changes. These 
include: 


e reductions in departmental budgets; 


e restructuring of departments and 
programs; 

e reductions in the size of the public 
service; and 


e new and tighter systems for 
managing spending. 
4.16 Subsequent to our 1993 
government-wide audits of internal audit 
and program evaluation, the Treasury 
Board issued its new Review Policy. The 
stated objective of this policy was to 
ensure that the government: 


e has timely, relevant and 
evidence-based information on the 
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No single factor is 
more important to 
effective internal audit 
than the attitude and 
expectations of the 
deputy head. 


Exhibit 4.1 


Organizational Responsibilities 
for Internal Audit 


performance of its policies, programs and 
operations, including the results they 
achieve; and 


e uses this information to improve the 
management and cost effectiveness of 
policies, programs and operations, and to 
account for results. 


4.17 The Treasury Board established 
the Review Policy as an umbrella policy 
that recognized three components: 


e management’s efforts and practices 
for monitoring, measuring and reporting 
on their accomplishments against stated 
objectives; 


e internal audit that provides senior 
management with independent, objective 
assurances and advice on departmental 
systems and management practices; and 


e program evaluation, which provides 
senior management with information on 
the extent to which policies and programs 
are relevant and cost-effective. 


Observations 


4.18 In this audit, we examined the 
extent to which departments and agencies 
have responded to the observations and 
recommendations in our 1993 Report. 
Further details on the audit objectives, 
scope and criteria are included at the end 
of the chapter in the section “‘About the 
Audit”. 


Stronger Support and Direction 
Needed from Senior Management 
4.19 Probably no single factor is more 


important to effective internal audit than 
the attitude and expectations of the deputy 


Organization/Unit Responsibilities 
Treasury Board Approving internal audit policy 


Treasury Board Secretariat 


Providing leadership and support to the internal 
audit community 


Monitoring the implementation and effectiveness 
of the internal audit policy, standards and code of 
ethics 


Departmental Deputy Heads 


Effective management of departmental programs 
and operations 


Appointing and providing support to a 
departmental head of internal audit who is 
independent of departmental operations 


Approving planned internal audit coverage 


Reviewing completed audit reports and 
management’s corrective action plans 


Using audit findings, analyses, conclusions and 
recommendations for decision making 


Heads of Internal Audit 


Managing the internal audit function in accordance 
with government policies and standards 
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head. Our 1993 Report noted this fact. 
Similarly, our study of best practices in 
other organizations stressed that strong 
support from senior management is an 
essential precondition to an effective 
internal audit function. Within a 
department, only the deputy head can 
ensure that internal auditors have the 
reporting relationship that allows them to 
remain independent from departmental 
operations. That independence is essential 
to maintaining their objectivity in carrying 
out their audit work and reporting the 
results. Further, only the deputy head can 
signal the importance of internal audit by 
becoming personally involved in 
establishing its mission, setting its 
priorities, reviewing results, and meeting 
regularly with the auditors. 


4.20 The key role to be played by 
deputy heads of federal departments and 
agencies is underscored by a significant 
difference in practice between public and 
private sectors. In the private sector, the 
audit committees of publicly held 
organizations are usually composed of 
non-employee members of the board of 
directors. This practice provides the basis 
for the independence of audit activities 
from the operations of the organization. In 
contrast, federal departmental audit 
committees are normally composed of 
managers of the department. 


4.21 The government’s internal audit 
policies have long recognized the 
importance of strong senior management 
support for internal audit. In fact, the 
government’s 1994 Review Policy 
identifies specific responsibilities in 
relation to internal audit for which deputy 
heads are accountable. These include 
determining what is to be audited, using 
the results of internal audits to maintain 
and improve programs, and maintaining 
strong internal audit and evaluation in 
their departments. 


tt essoeesiooeenancnnneceesc cee 
Internal Audit in Departments and Agencies 


4.22 In 1993, we reported that several 
senior managers said that they had an 
internal audit group in their department 
only because Treasury Board policies 
required it. This attitude was due, in part, 
either to a lack of understanding of the 
benefits that an effective internal audit 
unit can bring to a department or to prior 
bad experiences with the function. 


4.23 Three years later, we found the 
situation essentially unchanged in many 
departments. In some departments, the 
deputy heads were not involved in 
deciding what audits would be done; the 
choices were left to lower-level managers 
in the department. As a consequence, 
actual audit coverage may not address 
areas of most significance and risk or 
areas of most importance to senior 
management. 


4.24 We also noted that, even though 
regular follow-up to determine whether 
managers had acted on audit 
recommendations is required by Treasury 
Board Standards, it was not carried out in 
four of the departments that we reviewed 
in detail. 


4.25 During the audit, we reviewed 
senior management support and direction 
for internal audit units in a number of 
ways: 


e the frequency of meetings between 
the audit executive and the deputy head of 
the department; 


e the reporting relationship of the audit 
unit; 


e approval of planned audit activities 
by senior management; 


e consideration by senior management 
of the results of audits, and subsequent 
follow-up on planned corrective actions; 
and 


e the results of interviews with deputy 
heads and departmental managers. 
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A fundamental 
question is why, in 
general, there 
continues to be a lack 
of strong senior 
management support. 


We noted a strong 
correlation between 
those departments 
with strong senior 
management support 
and direction and the 
effectiveness of their 


internal audit function. 


4.26 During interviews conducted as 
part of our audit, deputy heads expressed 
general support for their internal audit 
function. However, few deputies or other 
senior managers could translate that 
support into specific expectations or 
requirements for internal audit services. 


4.27 In Public Works and Government 
Services Canada, the deputy head noted 
that having an effective internal audit 
group was not a question of policy; rather 
it was a question of good management 
practice. In this instance, the deputy noted 
specific uses he made of the function. For 
example, internal audit provided 
assurance, prior to signing significant 
contracts, that departmental contracting 
procedures had been followed. In another 
department, the audit committee (a 
requirement for all departments and on 
which the deputy head and other senior 
managers sit) had not met in over three 
years, either to review and approve 
planned audit coverage or to consider the 
results of audit work performed and the 
necessary corrective action. In this 
department, the internal audit group had 
completed very few audits. 


4.28 As was the case in 1993, we 
noted a strong correlation between those 
departments with strong senior 
management support and direction and the 
effectiveness of their internal audit 
function. 


4.29 One important way for 
departments to signal the importance of 
the audit function, and support for it, is 
through having internal audit report 
directly to the deputy head. However, of 
the seven departments reviewed in detail, 
only Public Works and Government 
Services Canada benefited from this 
reporting relationship. Accordingly, 
internal audit’s ability to examine 
departmental activities independently and 
objectively could be hindered. In one 


department, the organizational status of 
the head of audit was not sufficient, in our 
opinion, to permit the accomplishment of 
audit objectives. 


4.30 A fundamental question is why, 
in general, there continues to be a lack of 
strong senior management support. During 
interviews, managers often cited reasons 
similar to those noted in our 1993 Report. 
These included a lack of understanding of 
the function and poor results from internal 
audit. We also noted that government 
managers had a narrower range of 
expectations for the role of internal audit 
than managers in the private sector. 


4.31 We compared the internal audit 
expectations of senior managers in the 
Canadian federal government with those 
of senior managers in the private sector 
and other governments, using the results 
of surveys conducted by the Institute of 
Internal Auditors. The results are shown in 
Exhibit 4.2. There are sharp differences 
between the two groups surveyed with 
respect to internal auditing of efficiency 
and effectiveness, human resource 
management issues, and proactive 
initiatives. In each case, very few federal 
government managers reported that they 
expected internal audit to address those 
areas, in contrast to the majority of 
non-government managers. 


4.32 The difference in expectations 
with respect to human resource 
management issues may be due to the fact 
that the Public Service Commission 
conducts audits of human resources in all 
departments, so senior managers do not 
perceive a role for internal audit in this 
area. The differences with respect to 
efficiency and effectiveness and proactive 
audit work appear to be attributable to an 
incomplete understanding of the 
potentially valuable role of internal audit 
in these areas — a role identified for 
internal audit in the government’s policy. 
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Another factor may be that management 
looks to other sources for information 
concerning these issues. For example, the 
work of program evaluation includes 
coverage of operational issues. The impact 
of these factors can be seen in the limited 
internal audit coverage of some 
departmental activities, as described in the 
following section. 


4.33 The results of the 1995 Global 
Audit Information Network (GAIN) 
survey, which covers a broad range of 
industry groups (although no Canadian 
government departments participated), are 
shown in Exhibit 4.3. The results reflect 
the expectations of senior management for 
internal audit of organizations 
participating in the GAIN survey. These 
managers have high expectations that 
internal audit resources will be 
strategically allocated to address all 
aspects of their organization’s operations. 


4.34 There is a need for senior 
government managers to develop a more 
complete understanding of how internal 
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audit can best serve their organizations in 
order to get the greatest payback from the 
function. Because this is an issue that 
exists in many departments, it is important 
that leadership for improved 
understanding come from the Treasury 
Board Secretariat. 


4.35 As we reported in 1993, a 
pressing need remains for the Treasury 
Board Secretariat, together with senior 
managers, both to develop a common 
vision of the role of internal audit within 
the federal government and to decide how 
to obtain more value from the $48 million 
annual investment. 


Internal Audit Coverage 


4.36 We assessed the adequacy of 
internal audit’s coverage (that is, what the 
audit looked at) in terms of the extent to 
which audits focussed on: 


e high-risk areas; 


¢ areas of high priority for departments 
and the government; 


Institute of Internal Auditors 
Global Audit Information 
Network 
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A pressing need 
remains for the 
Treasury Board 
Secretariat, together 
with senior managers, 
to develop a common 
vision of the role of 
internal audit. 


Exhibit 4.2 
How Senior Management 


Perceives the Role of Internal 
Audit 


Financial/Administration 


Efficiency/Effectiveness 


Human Resource Management 


Proactive 
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The best audit work 
and best audit reports 
will be of little use if 
they do not deal with 
what is important. 


Exhibit 4.3 


Role of Internal Auditing - As 
Viewed by Audit Committees 
of Organizations Participating 
in the Institute of Internal 
Auditors Global Audit 
Information Network 


¢ significant computer systems; and 


e the areas identified by Treasury 
Board Internal Audit Standards as 
constituting the scope of internal audit 
coverage. 


Deciding what to audit — risk 
assessment 


4.37 The responsibility for 
determining what is to be audited, and 
when, ultimately rests with the deputy 
head. While departments are to determine 
what is to be audited, evaluated or 
otherwise reviewed, the Treasury Board is 
to identify government-wide issues related 
to its needs, or those of other Cabinet 
committees. The Treasury Board 
Secretariat is also responsible for the 
extent to which significant policies, 
programs and operations are audited over 
time. Since 1993, Treasury Board 
Secretariat has directed departments to 
conduct audits of year-end spending and 
advance payments. 


4.38 To get the most from the 
resources used to carry out internal audits, 
it is important that these audits focus on 
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the highest-priority and highest-risk areas 
of government and in individual 
departments and agencies. The best audit 
work and best audit reports will be of little 
use if they do not deal with what is 
important to departmental or government 
success. 


4.39 In 1993, we noted that 
improvements were still needed in audit 
coverage. As well, in some departments, 
the internal audit group did not undertake 
independent assessments of proposed 
coverage, limiting audits to those issues 
identified by departmental managers. We 
also noted that many internal audit reports 
contained observations of little 
significance. 


4.40 Our 1996 audit found that in 
general, departmental audit groups have 
not developed and implemented formal 
mechanisms for providing independent 
advice to deputy heads on the coverage 
and timing of audits. 


4.41 Advice on audit coverage needs 
to be based on an assessment of risk, a 
process through which professional 
judgment is applied in considering basic 


Compliance Human Resource 
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questions about departmental programs 
and operations. These questions include: 


e What can go wrong? 


e What is the probability of it going 
wrong? 


e What are the likely consequences if 
it does go wrong? 


e Who will be affected if it goes 
wrong? 


e What procedures has management 
implemented to control risks and to 
mitigate potential consequences? 


4.42 These are the types of questions 
that senior management should expect 
internal audit to address in developing and 
presenting annual audit plans. For the 
period covered by our audit, we noted 
that, in two of the seven departments we 
reviewed in detail, annual audit plans 
based on the degree of risk associated with 
a given audit area were neither prepared 
nor presented to senior management. 


Program review — a priority in all 
departments 


4.43 Program review requires that 
departments assess their programs to 
identify opportunities to increase 
efficiency and operate more economically. 


4.44 We would have expected internal 
audit groups to be widely used as 
analytical support in the reviews. 
However, very few of the audit groups we 
looked at were used by their departments 
as sources of information and advice 
concerning potential increases in 
efficiency or economy in support of the 
program review exercise. 


Internal audit coverage of computer 
systems is improving 


4.45 In investigating the internal audit 
coverage of automated systems, we first 
reviewed the extent to which internal 
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audits had been completed for the 23 
largest systems being developed across all 
government departments, as identified by 
the Treasury Board. These large systems 
are important for two reasons: 


e They are costly for the government 
to develop. 


e They are critical to the delivery of 
government programs. 


4.46 Continuously reviewing systems 
while they are under development can 
alert senior managers to changes in the 
factors relating to costs, anticipated 
benefits, controls or other issues in time to 
make adjustments if needed. It is 
generally recognized that changes to a 
system are much less costly in the early 
stages of design than after the system has 
become operational. We found that of the 
largest systems under development (some 
of which had been under development for 
over 10 years), approximately half had 
been audited once. We have therefore 
concluded that internal audit coverage of 
computer systems under development has 
improved since our 1993 audit. 


Internal audit coverage 
of computer systems 
under development 
has improved since 
our 1993 audit. 


4.47 We also examined the internal 
audit coverage of major operational 
computer systems in a sample of 
departments. Coverage of these systems 
varied considerably, from little or no 
coverage in some departments to more 
extensive coverage in others. 


Meeting Treasury Board Standards 


4.48 An internal audit can deal with 
many different subjects. Treasury Board’s 
Internal Audit Standards indicate that the 
scope of internal audit work should 
include: 


e the examination and evaluation of 
the adequacy and effectiveness of the 
department’s system of internal control 
(the management framework, which 
consists of the collection of inter-related 
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Exhibit 4.4 


Effective Use of Benchmarking 
in Internal Audit Reports 


systems and practices for achieving 
established objectives); 


e accomplishment of established 
objectives and goals for operations 
(specifically whether operations and 
programs are being carried out as planned, 
and whether operations are consistent with 
goals and objectives); 


e reliability and integrity of 
information; 


e compliance with policies, plans, 
procedures, laws and regulations; 


e safeguarding of assets; and 


e economical and efficient use of 
resources. 


4.49 We would have expected that 
internal audits would focus on all these 
areas. In reviewing recent audit reports 
from 39 government organizations, we 
found the following: 


e The most frequently reported internal 
audit conclusions and recommendations 
relate to departmental management 
frameworks, particularly the need for 
plans, strategies and mission statements. 
In the 51 completed audits we reviewed, 
there are more than 80 recommendations 
that called for the organization to develop 
plans, strategies and mission statements. 


e More than half of the internal audits 
comment on the reliability and integrity of 
departmental information. 


e About 75 percent of internal audits 
cover, to varying degrees, compliance of 
departmental operations with applicable 
legislation, policies or regulations. Only 
rarely did these audits question the 
administrative costs associated with 
implementation of these policies. 


¢ Very few internal audits report on the 
achievement of established objectives and 
goals. 


e Many audits deal with the means 
used to safeguard government assets, 
whether financial, physical or information. 


e Only about six percent of the audits 
we reviewed identified specific dollar 
savings, or efficiency improvements for 
departmental programs. In some instances, 
internal audits do identify changes that 
could lead to generally minor, unspecified 
increases in productivity or reduced 
operating costs. In two cases, the 
departmental audit units used benchmark 
comparisons of similar organizations to 
identify and recommend potential 
improvements in productivity. An 
example of such a case is described in 
Exhibit 4.4. 


4.50 These results are consistent with 
the expectations noted earlier of the 
federal government’s departmental senior 
managers in relation to the role and 
function of internal audit. While there is 


administrative costs in all offices. 


productivity. 


A federal agency was looking for ways of reducing its administrative costs in regional offices. 
In this instance, senior management asked the agency’s internal auditors to examine the issue. 
The auditors used a technique called “benchmarking”, which compares performance between 
similar operations. They collected workload and resource information on administrative 
activities in each region, and determined that costs per client served varied by up to almost 
100 percent among regions. Based on this analysis, they recommended a baseline standard for 


The auditors also identified a number of “best practices”, or innovations developed by 
individual regional offices to increase productivity, which could be used by all. 


This is an example of how effective internal audit work can assist an organization to enhance 
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considerable coverage of financial and 
administrative systems, there is only 
limited coverage of issues related to 
efficiency and achievement of operational 
objectives. In the private sector, senior 
managers expect internal audit to assess 
issues related to operational efficiency and 
the achievement of operational goals. 


4.51 We compared the results of our 
above-noted analysis of internal audit 
coverage in the Canadian federal 
government with the benchmark 
comparisons reported in the Institute of 
Internal Auditors (IIA) Global Audit 
Information Network. The results of this 
comparison, which are presented in 
Exhibit 4.5, indicate that internal audit 
within the federal government is 
comparable to the ITA database in terms of 
the amount of work done in evaluating 
internal control systems and assessing 
compliance with applicable authorities. 
However, internal audit within the federal 
government generally reports much less 
often on issues related to operational 
efficiency or achieving established 
operational goals and objectives. 
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4.52 It is possible that the cause of the 
relatively infrequent review of goal 
achievement by government internal 
auditors is the absence of goals and 
objectives against which to measure and 
report. 


Measuring the Performance of 
Internal Audit 


4.53 In examining this area, we 
assessed the extent to which departments 
were using appropriate methods and 
techniques for measuring the effectiveness 
of internal audit — that is, its contribution 
to improving the way a department 
operates. 


4.54 In 1993, we noted that virtually 
no internal audit group in government had 
a mechanism for measuring its 
performance on a continuing basis. This 
situation contrasted sharply with the 
performance measurement activities of the 
leading internal audit groups we identified 
in our 1993 study of best audit practices, 
Internal Audit in a Changing Management 
Culture. 


4.55 In response to our 1993 Report 
chapter, a committee with departmental 
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Departmental internal 
audit units have made 
some progress since 
1993 in measuring 
their performance. 


Exhibit 4.6 


Use of Customer Satisfaction 
Surveys to Measure Internal 
Audit Performance 


[_] Informal Measures 
] Formal Measures 
Pe No Measures 


and Treasury Board Secretariat 
representation developed and proposed 
various ways of measuring internal audit’s 
performance. We found that departmental 
internal audit units have made some 
progress since 1993 in measuring their 
performance; approximately half of the 
units now measure their performance in 
some way, most frequently through the 
use of customer satisfaction surveys. 
Using the results of data collected during 
our audit, we compared the frequency of 
use of such surveys by Canadian 
government departments with that of 
participants in the Institute of Internal 
Auditors Global Audit Information 
Network. The results of this comparison, 
which are reflected in Exhibit 4.6, indicate 
a higher proportion of GAIN participants 
who use formal performance measurement 
techniques. 


4.56 We have concluded that work 
remains to be done at the departmental 
level before all departments can obtain the 
benefits of complete and reliable 
information on internal audit’s 
performance, but progress is evident. We 
have also concluded that much remains to 
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be done at the central agency level. We 
found that the Treasury Board Secretariat 
has not reviewed either the departmental 
implementation of its audit policy and 
standards or the effectiveness of internal 
audit. 


4.57 As was true in 1993, the Treasury 
Board Secretariat still needs to carry out 
or facilitate periodic quality assurance 
reviews to obtain information on the 
performance of the various internal audit 
groups and to assess the appropriateness 
of its standards and internal audit policy. 


The Need for More Treasury Board 
Leadership Remains 


4.58 Our 1993 Report on internal audit 
noted major concerns with the then Office 
of the Comptroller General’s leadership 
for the internal audit function. 


4.59 The Treasury Board Secretariat 
reports that it has adopted an approach to 
facilitate the implementation of the 
Review Policy that is based on ongoing 
communication with departmental internal 
groups. Such activities include: 


e establishment of an advisory group 
of heads of review, designed to address 
issues of community-wide concern; 


e establishment of working groups to 
focus on such issues as the audit of 
information technology and means for 
measuring the performance of internal 
audit; 


e support for the Review Network, a 
group established to provide a means for 
sharing information and initiating projects 
of community-wide interest; 


e the preparation of the report, 
Strengthening Government Review, to 
describe how government is managing, 
using and strengthening review; and 


e in preparing the Treasury Board 
President’s Report on Strengthening 
Government Review, a request for deputy 
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heads to provide to the Secretariat 
information on how they managed the 
Review Policy within their departments. 


4.60 We found that very little progress 
has been made since 1993 in the following 
areas: 


e The Treasury Board Secretariat has 
not established effective mechanisms for 
assessing either the extent to which the 
objectives of the internal audit policy and 
standards are being achieved or whether 
departments and agencies have effective 
internal audit functions. The Treasury 
Board Secretariat has not acted to 
establish, co-ordinate or direct quality 
assurance reviews of internal auditing in 
departments and agencies. 


e The Treasury Board Secretariat has 
done little work with senior departmental 
managers to define, develop and 
implement effective internal auditing. 


e The need remains for effective 
mechanisms to identify and assess 
government-wide areas of risk. 


e Departmental internal audit 
managers continue to see a need for more 
functional leadership by Treasury Board 
Secretariat. 


4.61 In our view, the Treasury Board 
Secretariat needs to concentrate its efforts 
on communicating with deputy ministers 
about the role of internal audit and how it 
can be used more effectively, as well as on 
assessing the effectiveness of internal 
audit in departments. To carry out these 
two tasks effectively, the Treasury Board 
Secretariat must obtain and use staff who 
have extensive and recent experience in 
internal auditing. 


The Use of Internal Audit Results 


4.62 ___ If internal audits are to be 
effective, they must produce results that 
help departments improve their 
operations. As part of our follow-up, we 
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examined the extent to which departments 
implemented audit recommendations in a 
sample of seven departments by surveying 
departmental managers to whom audit 
recommendations had been addressed. 


4.63 Departmental managers reported 
that approximately 60 percent of the 
sample of internal audit recommendations 
had been implemented. In the remaining 
40 percent, almost half of the 
recommendations were not implemented 
due to changes in circumstances, such as 
reorganization or amalgamation of 
government departments or other events 
that resulted in the responsible managers 
no longer considering the 
recommendations appropriate. In the 
remaining cases, approximately 

20 percent of the total, no action had been . 
taken on the recommendations, for 
reasons we were unable to determine. 


4.64 According to Treasury Board’s 
Internal Audit Standards, departmental 
internal audit groups are to determine and 
report how managers have responded to 
reported audit findings. Although most 
departments do carry out some form of 
follow-up, some do not, or do so only at 
the request of departmental managers. In 
these cases, there is no mechanism for 
regularly informing deputy heads on how 
the department has dealt with internal 
audit findings. 


Concerns over the Review Policy 


4.65 The biggest single change since 
1993 that has affected internal audit has 
been Treasury Board’s Review Policy. 
This new policy maintains the previous 
requirement for internal audits of 
government programs and operations in 
all departments and agencies. It also 
designates the deputy head as the person 
principally accountable for carrying out 
the policy. The main changes from the 
previous (1992) policy are to include 
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The Review Policy is 
not clear on the role 
that internal audit is 
expected to play. 


within the concept of review the efforts 
and practices established by individual 
line managers for assessing the operations, 
programs and policies that they manage. 
As well, the new policy establishes an 
enhanced requirement to make 
information from all reviews more readily 
available to the public. 


4.66 In making these changes, the 
Review Policy has altered the traditional 
understanding of the concept of review. 
Traditionally, review has referred to the 
professions of internal auditing and 
program evaluation. While internal audit 
and program evaluation are different 
management tools, each having its own 
objectives, they are both characterized by 
their objectivity and independence from 
the operations, programs or policies that 
they are assessing. Both are also 
conducted in accordance with recognized, 
established standards. 


4.67 Clearly, management’s own 
efforts to establish goals and objectives 
and to measure and report on their 
accomplishment are important and 
fundamental to good management. 
However, the inclusion of traditional 
management activities within the concept 
of review does not recognize the basic 
distinction between management and the 
independent assessment functions of 
internal audit and program evaluation. 


4.68 The impact of this lack of clarity 
within the policy framework is illustrated 
in the President’s Report on Strengthening 
Government Review. This report provides 
a list of completed audits, evaluations and 
management-led reviews, all of which are 
described as coverage of federal programs 
and policies. There is no distinction 
between quality and degree of assurance 
provided from the results of 
management’s own efforts to measure and 
report on its performance and the 
assurance provided by the independent 


and objective reports of internal audit and 
program evaluation. 


4.69 The Review Policy is not clear on 
the role that internal audit is expected to 
play. It variously describes audit’s role as 
follows: 

e “independent and professional advice 
and assurances on the performance of the 
management framework....” related to cost 
effectiveness of program delivery and 
internal operations, the economy, 
efficiency, and protection of resources, 
information integrity and compliance with 
statutes and policies; 

e “...whether the management 
framework is well designed and is 
working as intended,” and whether 
program delivery activities and internal 
operations are carried out cost effectively; 
and 

e accomplishment of established 
objectives and goals for operations. 


4.70 In other parts of the Review 
Policy, the definition of review activities, 
including internal audit, is different: the 
key issues to be reviewed are “success”, 
“cost effectiveness” and “relevance”. 
Further on in the policy, there is reference 
to another 12 “attributes of effectiveness”, 
which are suggested as guidelines for 
conveying a fair and complete picture of 
performance. 


4.71 The net result of these differing 
statements is that, at the most fundamental 
level, the new policy has introduced 
confusion as to what internal audit should 
be doing. Should managers expect that 
auditors will provide information on the 
extent of operational effectiveness in the 
department, as required by the Treasury 
Board Internal Audit Standards? Or is this 
issue to be addressed elsewhere, since it is 
not part of the internal audit role defined 
in the policy statement? Accordingly, the 
Treasury Board needs to clarify the 
understanding of internal audit in the 
federal government. 
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Access to Information Affects 
Internal Auditing 


4.72 Nearly all internal audit 
managers with whom we met indicated 
that departmental internal auditing is 
affected by the laws and policy relating to 
access to information. The requirement to 
make the results of audit work available to 
the public affects the timing of audit work, 
the issues addressed and the reporting of 
the results. Government departments, like 
all organizations, are not anxious to make 
their bad news public. Access, however, is 
not an issue that arises for 
non-government internal auditors because 
there is no legislated right of public access 
to private sector internal audits. We noted 
that the Treasury Board Secretariat has 
recently initiated a study to review the 
effects of openness on internal audit. 


Conclusions and 
Recommendations 


4.73 Internal audit is a fundamental 
management tool that can provide senior 
management with assurances about the 
efficiency and effectiveness of key 
financial, administrative and operational 
activities and departmental practices, 
along with suggestions for improvement. 


4.74 In 1993, we found a great deal of 
variability in the effectiveness of internal 
audit in federal government departments 
and agencies. The overall situation 
remains essentially unchanged today. 
Although some departments have 
improved, others appear to be less 
effective. 


4.75 We have identified a number of 
factors that are common to effective 
internal audit units in the federal 
government. Taken together, addressing 
the factors listed in our recommendations 
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will increase substantially the probability 
that internal audits will deal with 
important issues, and bring the greatest 
benefits to organizations. 


4.76 The federal government should 
direct efforts to achieving the following 
characteristics associated with effective 
internal audit: 


e a deputy head of the department or 
agency who, in balancing the many 
priorities that he or she faces, is 
supportive of internal audit and 
demonstrates that support through a 
clear, articulated, personal point of view 
on the role and benefits of internal 
audit; and who: 


e maintains regular and frequent 
communication with the head of 
internal audit; 


¢ communicates to departmental 
managers the role of internal audit 
as an independent source of 
information and improvement; and 


e follows up on the results and 
recommendations from audits; 


e direct reporting of the 
departmental internal audit unit to the 
deputy head; 


¢ audit methodology that provides 
selective, risk-based coverage and 
focusses on areas critical to 
departmental success; 


e aclear mandate or charter for 
departmental internal auditing that 
provides for the independence of 
internal audit consistent with internal 
audit standards; 


e expertise in the use, control and 
auditability of the organization’s 
technology; and 


e ameans of measuring internal 
audit’s effectiveness as a basis for 
identifying and assessing improvements 
to internal auditing. 
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4.77 In order to emphasize the role 
and independence of internal audit, the 
federal government should implement 
the private sector practice of appointing 
members who are not public servants to 
its departmental audit committees. 


4.78 An essential first step to 
achieving and communicating 
accomplishments against stated policy 
objectives is a clearly articulated policy 
framework. In this respect, Treasury 
Board Secretariat, in concert with key 
stakeholders, should initiate efforts to 
clarify the role of internal audit. The 
Review Policy should clearly recognize 
the distinction between the degree of 
assurance provided by management-led 
reviews and that provided by the 
independent and objective analysis of 
internal audit. 


Treasury Board Secretariat’s response: 
We agree that strong internal audit is an 
important management tool within the 
spectrum of review. Good information, 
which can come from internal audit, does 
help to deliver quality and cost-effective 
services to Canadians. Investing resources 
to obtain this information must receive the 
full support of deputy ministers and other 
senior managers. 


With reduced resources and dramatic 
changes in the functions and structures of 
government, public servants are learning 
how to better use the resources and tools 
available to them. 


Within this context, and over the past two 
years, the Treasury Board Secretariat has 
focussed on improving the information 
that is available on the performance of 
government policies, programs and 
operations. The availability and 
accessibility of such information is seen as 
essential in improving the accountability 
processes of government and in helping 
managers make the difficult decisions they 


face. 


In June 1994, a new Review Policy was 
issued by Treasury Board. The policy 
promotes the availability of relevant and 
evidence-based information on 
performance. A key feature of the new 
policy was the recognition that there are 
several sources of information available to 
decision makers. These include 
information from internal audits, program 
evaluations and other reviews. 


As an integral part of the Review Policy 
package, a revised policy on internal audit 
was approved. This policy clearly sets out 
the elements that are required for an 
effective internal audit function in 
government; the policy demonstrates the 
government's commitment to maintaining 
strong internal audit. 


The new policy package, which also 
included a policy on evaluation, was 
intended to allow departments to select 
the right review tool for the job. These 
changes complement the new Expenditure 
Management System and departmental 
business plans, as well as the 
government s initiatives to improve 
reporting to Parliament and to make 
greater use of information technology to 
streamline administration and improve 
delivery of service. Readers of this 
chapter should consider what is being 
reported about internal audit in the 
context of the set of management reforms 
being put in place. 


In November 1995, the President of the 
Treasury Board tabled in Parliament his 
first annual report on review, entitled 
Strengthening Government Review. The 
preparation of this report involved 
monitoring departmental audit, evaluation 
and review activity and was the first 
report to Parliament, by the government, 
to describe its progress in encouraging the 
development and use of performance 
information. 


The approach of the Treasury Board 
Secretariat to monitoring focusses on the 
products and results of reviews, rather 
than the more process-oriented approach 
suggested in this chapter. The 
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results-oriented approach considers all 
sources of information based on a variety 
of interactions with departments and 
recognizes the strengths and weaknesses of 
the various tools available. 


Substantial progress has been made over 
the last two years in promoting an 
understanding of the new Review Policy, 
which promotes the principles of 
results-based management, effective 
control and clear accountability. Our 
framework reflects public service ethics 
and values. There have been improvements 
in the information that is available to 
managers, Parliament and the public. 
Further progress is needed. A specific 
action plan was set out in Strengthening 
Government Review and the government 
is committed to the initiatives in this 
action plan. 


a 


Internal Audit in Departments and Agencies 


With reference to the 1993 audit, we have 
made significant progress, specifically in 
the audit of information technology 
systems; in developing an approach for 
the measurement of performance of 
internal audit units in departments; in 
communication of internal audit best 
practices; and in improving information to 
Parliament on audit work that is 
completed, under way and planned. 


In the coming year we will work with 
internal auditors and the Auditor General 
as well as the other professionals in the 
review community to provide better 
guidance on the range of tools within the 
spectrum of review. This guidance will 
include, as part of our approach to 
successful review, the contribution of 
internal audit in improving the 
accountability and management of 
government operations. 
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Background 


About the Audit 


In 1993, we reported the results of our audit of internal auditing in the Government of Canada. At that time, 
we concluded that the quality of internal auditing varied considerably from department to department. We 
noted the importance of senior management support and that internal auditing was not gaining ground in 
closing the gap between what might reasonably be expected from the function and its actual performance. We 
also noted that it was too early to assess a number of initiatives that the then Office of the Comptroller 
General had taken to improve internal audit’s performance. 


Objectives 


The objective of our current audit was to determine the extent to which departments and agencies have 
responded to the observations and recommendations set out in our 1993 chapter in five areas: 


the level of management support provided to internal audit; 
e audit coverage, that is, the departmental operations that internal audit examined; 
e how internal audit groups were measuring their own performance and contribution to the organization; 


e the extent to which the Treasury Board Secretariat was fulfilling its role as functional leader of the 
internal audit community — specifically by monitoring the effectiveness of government internal auditing 
policy, and assessing the effectiveness of the internal audit standards and the code of ethics in achieving 
stated policy objectives — as required by the Treasury Board Review Policy; and 


e the extent to which senior management was acting on the recommendations in internal audit reports. 


Scope and Approach 


As part of our audit we conducted a survey of the costs and practices associated with internal audit in all 

| departments and agencies. We then compared the results of this survey to the Institute of Internal Auditors 

_ Global Audit Information Network (GAIN), a database that compares internal audit costs and practices across 
| abroad range of internal audit groups, in both the private and various public sectors. 


person-years working in internal auditing and approximately 48 percent of the government’s 1995-96 internal 


| We reviewed in greater detail the internal audit units of seven departments that account for 61 percent of 
| audit budget. The departments included in our detailed review were: 


e Canadian International Development Agency 
| e Correctional Service Canada 

e Fisheries and Oceans Canada 

e Human Resources Development Canada 


e Public Works and Government Services Canada 
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e Revenue Canada 
e ‘Treasury Board Secretariat. 


To obtain a government-wide perspective on internal audit coverage and the types of observations, 
conclusions and recommendations being reported by internal audit groups, we also reviewed a sample of audit 
reports from all government departments and agencies. | 


Criteria 


Each section under Observations contains criteria specific to the particular topic. In more general terms, the 
criteria we used to assess the current state of internal audit were the Treasury Board’s internal audit policy 
and standards for the professional practice of internal audit, which are largely based on those of the Institute 
of Internal Auditors. 


Our criteria are also based on the study of best practices conducted by this Office as part of its 1993 
government-wide audit of internal audit. The study, entitled Internal Audit in a Changing Management 
Culture, covered 40 government and private sector organizations in Canada and the United States. It showed 
that internal auditors were changing in response to the evolving business environment. For example, the best 
internal audit units were becoming involved early in the development of computer systems, devoting more 
attention to deciding what should be audited, and working more closely with managers. 


Audit Team 


Brian Brisson 
Bob Cardillo 
Christine Kelly 
John Pritchard 


For information, please contact Bruce Sloan, the responsible auditor. 


Report of the Auditor General of Canada — May 1996 4-23 


: 
iV a 
7 
: * a 7 
ars et 


Ce CR a ee 


7 - . =—— Ga ry 
7s _ 
© 


| ‘ wove 
r<] hee ene Ee at 


s 
te 
: 
7 - 7 ni oS -~ 
a) ae 
' ee 
} 1 ye 
Pyael 1 
a a 
i 
‘ 
| 
‘i a 
“y : 5 
-s ‘4 7 ee UT 
ie 
‘ 
o y 
: a 
. 
- 
i 
e 
oe 
i - 
4 
. 
a , 
 - 
t 
o f 
: yy. i 
in 
‘ : 
a ; 
re) 1 
ee 


Report of the 
Auditor General of Canada to the House of Commons — 1996 


Table of Contents 


Volume 1 —- May 1996 


Chapter 

Foreword and Main Points 

1 Other Audit Observations 

pi The Implementation of Federal Environmental Stewardship 

3 Evaluation in the Federal Government 

4 Internal Audit in Departments and Agencies 

5 The Reform of Classification and Job Evaluation in the Public Service 
Peacekeeping 
Preface 

6 Foreign Affairs and International Trade Canada 

s | National Defence 

8 Canadian Security Intelligence Service — National Headquarters Building Project 

2 Agriculture and Agri-Food Canada — Animal and Plant Health: 
Inspection and Regulation 

10 Correctional Service Canada — Rehabilitation Programs for Offenders 

11 Revenue Canada — Combatting Income Tax Avoidance 


12 Veterans Affairs Canada — Health Care 


stint made Aaa 
c/n 
orang Seta 
ee Te ceieeiimme ae sy. : 
te. | eam: 


See ee re ea - 
- srthe lore: ©, ~ 


(eel? \ em irbe eaciall shine 


sit) ee sogheencemstinde § 
mdiblis Simm rer! 


CEP a 40 ine dae haere singed 2 sok pe temedrorees3 
ey es i (iiaag >) * dries) vanes 


D> C04 eG areTA oltre 


Order Form for 1996 Report Material 
If you wish to obtain other 1996 Report material, consult the Main Table of Contents and indicate the chapters and 
formats you wish to receive. Mail the form to: 
Office of the Auditor General of Canada 


240 Sparks Street, Stop 10-1 
Ottawa,ON K1A 0G6 


or FAX form to: (613) 954-0696 


Please Print 


Name: 


Organization: 


Address: 


City: Province: 


Country: Postal Code: 


Telephone: _ ( ) - 


Please Specify Chapter Number and Quantity 


Chapter(s) # ______ English ______ French 
e ____ English ____ French 
# _____ English _____ French 
# ______ English _____ French 
# English ______ French 
# _____ English ______ French 


Foreword and Main Points (May 1996) 
Chapter 1 Bilingual 


Diskette (3 1/2” HD) 
(May 1996) English French 


CD-ROM (contains Reports 
from 1987 to May 1996) Bilingual 


Video (contains Selected 
Highlights of the 
Report - May 1996) English French 


“The Report is also available on the /nternet at www.oag-bvg.gc.ca 


vapter 5 


Report of the | 
Auditor General 


of Canada | 
to the House of Commons 


_ Chapter 5 
The Reform of Classification and Job Evaluation 
in the Public Service 


May 1996 


Report of the 
Auditor General 


of Canada 
to the House of Commons 


Chapter 5 
The Reform of Classification and Job Evaluation 
in the Public Service 


May 1996 


This May 1996 Report comprises 12 chapters and a Foreword and Main Points. In 
order to better meet clients’ needs, the Report is available in a variety of formats. 
If you wish to obtain another format or other material, the Table of Contents and — 
the order form are found at the end of this chapter. 


© Minister of Public Works and Government Services Canada 1996 
Cat. No. FA1—1996/1-5E 

ISBN 0-662—24470-2 

Copies frangaises aussi disponibles 


Chapter 5 


The Reform of Classification and 
Job Evaluation in the 
Public Service 


The audit work reported in this Shapier was sonduetent in accontanee with the 
legislative mandate, policies and practices of the Office of the Auditor General. 
These policies and practices embrace the standards recommended by the Public 
Sector Accounting and Auditing Board : PSAAB B) af the Canadian Institute ee 
Chartered Accountants. 
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The Reform of Classification and 


Job Evaluation in the 
Public Service 


Assistant Auditor General: Maria Barrados 
Responsible Auditor: Jacques Goyer 


Main Points 


Sal The reform of classification and job evaluation was one of the important systemic changes associated 
with public service renewal. Because of important links with other systems such as labour relations and pay 
administration, classification and job evaluation have been considered the “cornerstone” of human resource 
management in the public service. 


3.2 In response to recommendations for change, the Treasury Board Secretariat undertook to develop a single 
job evaluation plan to replace the some 70 currently in use, and to reduce significantly the number of occupational 
groups and sub-groups into which jobs are classified. Given the difficult context, this was quite a challenging task. 


—) Based on our assessment of the June 1995 version of the Universal Classification Standard, our audit 
concluded that, after some six years, the challenge posed by the reform has been only partially met: 


e The new job evaluation plan developed by the Secretariat is a definite improvement and has many 
promising features, but a number of significant issues need to be addressed to ensure that it is a valid 
and reliable tool, capable of evaluating all jobs in the public service while meeting requirements for 
pay equity that stem from the Canadian Human Rights Act. 


e Opportunities exist to simplify further and/or to reduce the costs of classification and job evaluation 
and some of the related systems and practices, such as pay administration. 


5.4 While the Treasury Board Secretariat has undertaken the reform with a commitment to “cost neutrality’’, 
we found that the cost and benefits of the reform were not known with any degree of precision. 


os) At the conclusion of the audit the Secretariat was taking a number of important measures to address 
issues identified by the audit. The Standard has potential and can be made to work. Success will depend, however, 
on the willingness and ability of the stakeholders to work together to achieve the objectives of the reform at a 
reasonable cost to the taxpayer. 
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Introduction 


5.6 In 1989, the government 
launched Public Service 2000, an 
initiative to renew the public service. Its 
aim was to achieve a culture more 
oriented toward serving clients and 
delivering results. The government wanted 
to simplify the employment and personnel 
management regime so that the resources 
devoted to these activities could be 
redirected toward better service. A task 
force headed by a deputy minister and 
composed of senior managers was set up 
to review classification and job evaluation 
in the public service. 


The Current Classification and 
Job Evaluation Systems 


Sa Classification. In the public 
service, jobs are currently classified into 
72 occupational groups and 

106 sub-groups, and into a varying 
number of levels within these groups. 
Exhibit 5.1 provides examples. The 
current occupational group structure was 
created in the late 1960s, mainly to serve 
as a basis for bargaining units. The 
structure has remained essentially 
unchanged and continues to be the main 
framework for negotiating pay. As of 
July 1990, there were 78 bargaining units. 
As a result of these complex occupational 
group and bargaining structures, there are, 
according to Treasury Board Secretariat, 
currently some 840 pay rates and some 
70,000 rules governing pay and 
allowances in the public service. This 
impacts heavily on the complexity and 
cost of related administrative systems. In 
1994 we reported that, partly due to its 
complexity, the annual administrative cost 
of the federal pay system per employee 
was some 1.5 times higher than in 
provincial governments. In October 1995 
we reported that the project initiated to 
improve the pay process through better 
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use of technology was cancelled after 
more than $60 million of the estimated 
$120 million total cost of this project had 
been spent. The complexities of the pay 
system were a significant factor in the 
difficulties encountered in that project. 


5.8 Job evaluation. Job evaluation is 
the process used to determine the relative 
value of jobs, based on content and 
requirements. Jobs are evaluated using 
factors and rating guides (the job 
evaluation plan) and reference 
(benchmark) positions contained in a 
“classification standard”. The resulting 
evaluations are used to determine which 
jobs within the organization should 
receive the same rate of pay and which 
should be paid at different rates. 
Approximately 70 different classification 
standards are used in the public service. 


A complex and lengthy process 


ee) In the public service, 
classification and job evaluation are 
intertwined. The process is both complex 
and lengthy, and thus costly. It requires 
several steps, including: 


e gathering and reviewing information 
about a job; 


¢ writing a detailed and thus lengthy 
job description with appropriate 
organizational charts that accurately 
depict the job and its relation to other jobs 
in the organization; 


e determining the occupational group 
of the job by assessing how the job 
description fits, or does not fit, globally 
with definitions contained in various 
classification standards; 


e evaluating the requirements of the 
job, using the job evaluation plan and the 
reference positions (benchmarks) 
contained in the applicable classification 
standard to determine the value of the job, 
usually expressed in points; and then 


e determining the level of the job by 
comparing its point value to the 


The government 
wanted to simplify the 
employment and 
personnel 
management regime 
so that the resources 
devoted to these 
activities could be 
redirected toward 
better service. 


In the public service, 
Classification and job 
evaluation are 
intertwined. The 
process is both 
complex and lengthy, 
and thus costly 
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predetermined range of points established 
for each of the levels in the occupational 
group. Exhibit 5.2 illustrates the factors 
currently used to evaluate positions in the 
Secretary sub-group of the Secretarial, 
The current Stenographic and Typing occupational 
pee : group and the maximum number of points 
classification and job allocated to each. It also shows the point 
evaluation systems ranges set for each level of the sub-group. 
Exhibit 5.3 presents the rating guide used 
cannot be used to evaluate the factor “consequences of 


effectively to achieve error” for the Secretary sub-group. 
or maintain pay equity. Pay equity requirements: more 
complexity 


5.10 The concept of “equal pay for 
work of equal value” has had the force of 
law in organizations under federal 
jurisdiction — including the public 
service — since 1978. Such organizations 


Exhibit 5.1 


Examples of Current Occupational Groups and 
Sub-Groups with the Number of Levels for Each 


must now be able to compare the value of 
jobs in predominantly female groups with 
jobs in predominantly male groups, using 
a common standard and taking into 
account four factors specified in section 
11 of the Canadian Human Rights Act: 
responsibilities, skills, effort and working 
conditions. 


5.11 Developed in the late 1960s, the 
current classification and job evaluation 
plans were not designed to compare the 
value of jobs in one occupational group 
with those in another group. In addition, 
current job evaluation plans do not always 
include the factors specified in the Act. 
The current classification and job 
evaluation systems thus cannot be used 
effectively to achieve or maintain pay 
equity. Several complaints have been filed 


Group and Sub-Groups 


‘Number of Levels 


Secretarial, Stenographic, Typing (ST) 
Typing (ST-TYP) 
Stenographer (ST-STX) 
Office-Composing — Equipment Operation (ST-OCE) 
Court Reporter (ST-COR) 
Secretary (ST-SCY) 


Clerical and Regulatory (CR) 

Program Administration (PM) 
Mediation/Conciliation Officer (PM—MCO) 

Administrative Services (AS) 


Education (ED) 
Language Teacher (ED-LAT) 
Elementary and Secondary (ED-EST) 


Education Services (ED—-EDS) 


Education Support (EU) 
Language Instructor (EU-LA) 
Physical Education Instructor (EU-PEL) 
Teacher’s Aid (EU-TEA) 


oO WOW YN FPN WHY WY 


2 
Experience — 
Education grid 
5 


* No levels 


Source: Report of the Task Force on Classification and Occupational Group Structures, July 1990 
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with the Human Rights Commission in 
relation to pay equity issues, and a Human 
Rights Tribunal has ruled that some 
practices are discriminatory. 


5.12 To deal with pay equity issues, 
complaints and rulings, the Treasury 
Board Secretariat has had to resort to 
using an additional, separate “common” 
job evaluation plan and a cumbersome 
system of equalization payments to 
female-predominant groups. This has 
added a significant layer of complexity to 
already complex job evaluation and pay 
systems. 


Why are classification and job 
evaluation important? 


5.13 There are very important and 
complex links between the classification 
of a position and other human resource 
management decisions or systems, such as 
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staffing, collective bargaining and pay 
administration. For these reasons, 
classification and job evaluation are 
considered the “cornerstone” of human 
resource management in the public 
service. For example, because 
appointments are usually made “to a 
position”, reclassifying a position to a 
higher level leads to initiating a staffing 
process to find a candidate or promote the 
incumbent. The resulting appointment will 
trigger a pay transaction. Exclusion from, 
or membership in, a bargaining unit will 
be also determined by the occupational 
group and level of the position and the 
duties performed by the incumbent. 
Because of these links, any change to the 
current classification and job evaluation 
systems, such as simplification, can have 
significant multiplying effects. The reform 
of classification and job evaluation was 
considered by many as one of the 


Factors Weights 
r + 
Knowledge 
Education 150 
Experience 260 410 
Complexity 250 
Consequence of Error 100 
Contacts 200 
Supervision 40 
1,000 points 
Point Boundaries 
= : 
Levels Points 
is 1 
ST-SCY 1 265-400 
ST-SCY 2 401-550 
ST-SCY 3 551-700 
ST-SCY 4 701-1000 


There are very 
important and complex 
links between the 
Classification of a 
position and other 
human resource 
management 
decisions or systems. 


Exhibit 5.2 


Factors Used to Evaluate Positions in 
the Secretary Sub-Group with Applicable 


Weights and Point Boundaries 


Source: Classification standard for the Secretarial, 
Stenographic, Typing Occupational Group, 


Treasury Board Secretariat, 1976 
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In 1990, we concluded 
that a fundamental 
change was 
necessary. 


Exhibit 5.3 


important systemic changes resulting from 
Public Service 2000. Exhibit 5.4 describes 
in more detail some of the links between 
classification and job evaluation and other 
aspects of human resource management in 
the public service. 


Previous Audits of Classification 
and Job Evaluation 


5.14 In our 1983 Report chapter titled 
“Constraints to Productive Management’, 
we noted that managers found it difficult 
and time-consuming to apply the current 
job evaluation plans. They felt it 
hampered productive management. In 
1984 we reported that job evaluation in 
the public service was subject to great 
pressures at a time when the overall size 
of the public service was being restrained. 


Examples of a Rating Guide for the Factor “Consequences of 
Error” for the Secretary Sub-Group 


We also noted significant deficiencies in 
the way the system was managed. In our 
follow-up report in 1986 we expressed 
concerns about the pace of Treasury Board 
Secretariat’s review of a significant 
number of outdated classification 
standards. In 1990, another follow-up 
noted the relatively high number of 
occupational groups in certain 
departments and agencies. We reported 
that six departments had at least 

40 occupational groups, and that one of 
them had 56. We concluded that a 
fundamental change was necessary. In our 
1994 Report chapter “Management of 
Scientific Personnel”, we recommended 
that consideration be given to creating a 
single occupational group for employees 
doing scientific research, to simplify the 


Consequences of Error and Degree Benchmark Position Points 
Errors would be minor in nature due to the type of work M-15.2* 
performed and may cause minimal loss of time for II-16.2 35 
correction, or may cause slight annoyance or inconvenience. IH-19.3 
A 
Errors would have moderate consequences, e.g. may result 
in some loss of time on the part of several employees for 111-203 
correction, may cause inaccuracies in records or reports and 11-223 
have a limited effect on activities based on these reports, U2 43 70 
may result in minor loss of goodwill, may be embarrassing 1-253 
to the individual for whom secretarial services are being ; 
provided or to others inside or outside the public service. 
B 
Errors would have serious consequences, e.g. may cause 
inaccuracies in reports or records and result in faulty 
information upon which decisions are made, may cause 
substantial delay or considerable loss of work time or 
| productivity, may have serious adverse effects on ee 100 
relationships with the public, may cause considerable loss ise 
of goodwill indicated by a series of complaints, may be 
embarrassing to the Department or to the government 
generally. C 


* Refers to specific benchmark positions also contained in the “classification standard” 


Source: Classification standard for the Secretarial, Stenographic, Typing Occupational Group, Treasury Board Secretariat, 1976 
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occupational group structure in research 
establishments. 


Proposals for Reform: Significant 
Change 


315 The task force set up under 
Public Service 2000 was charged to 
examine the occupational group structure 
and the administration of job evaluation in 
the public service. In its July 1990 Report, 
the task force recognized that both were 
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cumbersome, too complex, very 
time-consuming and too costly to operate, 
to say nothing of their impact on the costs 
of other human resource management 
systems. According to the Treasury Board 
Secretariat, in 1989-90 alone there were 
some 216,000 classification actions, more 
than half of them involving major 
changes. Yet it has been established that 
the total number of staffed “full-time 
permanent” positions in the public service 
at that time was only 181,355. Partly due 


Staffing 


position. 


profile. 


candidate or promote the incumbent. 


The classification of an individual employee is dependent on the level of the 


Statements of qualification used in staffing are derived from job descriptions. 
Any significant change in the job will necessitate a change in the selection 


Reclassification of a position leads to a formal staffing action to find a 


Deployments (transfers without appeal rights) can be made only if the 
classification and level of the employee corresponds to the classification and 
level of the position he/she is moving to. 


Labour Relations — 


occupational group basis. 


Bargaining units are established on the basis of the occupational group 
structure. There are currently some 80 bargaining units. 


Collective agreements (salary and working conditions) are negotiated on an 


Membership in a bargaining unit is determined by the classification and level 
of the individual and the duties performed. 


The abolition of a position can result in the employee being declared surplus 
and reassigned or eventually laid off. 


Pay 


There is a pay rate or salary scale for each level within an occupational group 
or within a sub-group. There are also annual increments embedded in each 
salary scale. There are currently some 840 rates of pay and an estimated 
70,000 rules governing pay and allowances in the public service. 


Collective agreements expire or come into effect at different times, leading to 
ongoing collective bargaining. 


Because some current job evaluation practices have been judged 


discriminatory, equalization payments have been granted to certain employees. 


Official Languages 


Language requirements must be established for every position created. 


In many instances incumbents need to be tested or re-tested to determine if 
they meet the requirements. Change in responsibilities or reclassification 
necessitates a re-examination of the requirements. 


The task force set up 
under Public Service 
2000 was charged to 
examine the 
occupational group 
structure and the 
administration of job 
evaluation in the 
public service. 


Exhibit 5.4 


Some Key Links between 
Classification and Job 
Evaluation and Other Human 
Resource Systems 
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The task force 
concluded that 
classification and job 
evaluation should be 
greatly simplified. 


Exhibit 5.5 


Guiding Principles for a New 
Classification and Job 
Evaluation System, as 
Proposed by the Task Force 


Source: Public Service 2000: 
Report of the Task Force on 
Classification and Occupational 
Group Structures, July 1990 


to the restructuring of departments, the 
Treasury Board Secretariat could not 
provide us with more recent information 
on the number of classification 
transactions over the last five years. 


5.16 The task force also pointed out 
that the classification and job evaluation 
systems posed many obstacles to 
redeployment of resources, and that 
changes in technology and work 
organization had made these systems 
obsolete in many cases. The task force 
noted that the current systems were highly 
susceptible to manipulation. 


5.17 The task force concluded that 
classification and job evaluation should be 
greatly simplified, and recommended that 
the current 72 occupational groups and 
106 sub-groups be reduced to 23 and 8 
respectively. It also recommended 
amalgamating 15 occupational groups into 
a single group, the General Services group 
(GE), which would include some 125,000 
public servants. The task force 
recommended that job evaluation and its 
administration be simplified by 
developing a single job evaluation plan. It 
suggested that job descriptions be no more 
than three pages long and ina 
computerized format wherever possible. 


— 
. 


Provide a productive working 
environment. 


Be simple and transparent. 
Be flexible. 


Stimulate career progression. 


wk Ww ON 


Provide equal pay for work of equal 
value. 


6. Establish better external relativities. 
7. Provide fairer internal relativities. 


8. Permit broader spans of control with 
fewer levels. 


9. Be suitable for management ownership. 
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Exhibit 5.5 presents the guiding principles 
that the task force proposed for the new 
job evaluation plan. 


5.18 In the November 1990 White 
Paper Public Service 2000: The Renewal 
of the Public Service of Canada, the 
government endorsed many of the task 
force recommendations. In January 1991, 
Treasury Board Secretariat started to 
implement them with a commitment to 
“cost-neutrality”, which was never clearly 
defined. 


Undertaking the Reform: A 
Formidable Challenge 


5.19 Following up on the 
recommendations of the task force and the 
White Paper, the Treasury Board 
Secretariat decided to develop, within a 
self-imposed timeframe of 24 months, a 
totally new job evaluation plan that would 
be capable of evaluating the job content of 
over 200,000 positions, widely different in 
nature and scope. The plan had to be 
suitable for use in more than 

50 departments and agencies whose size 
and activities vary greatly. Moreover, 
reducing the number of occupational 
groups meant dealing with several unions, 
some of which could eventually disappear 
with the reduction in the number 

of occupational groups — all of this at a 
time when labour relations were difficult, 
salaries had been frozen and the 
government and unions had been in 
litigation for several years over pay 
equity. This was quite a challenging task. 


5.20 Since 1991, the Treasury Board 
Secretariat, in collaboration with 
departments, has developed a new job 
evaluation system, the Universal 
Classification Standard. The Standard is 
intended to replace some 70 current 
classification standards, using a single 
approach to job evaluation. 
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5.21 Major changes, at times 
unexpected, interfered with the project — 
such as the June 1993 decision to 
restructure departments, and the 
downsizing of the public service by some 
45,000 positions over three years due to 
changes in government policies, programs 
and service delivery. 


S22 When we began our audit, in the 
fall of 1993, a “final draft” version of the 
Universal Classification Standard had 
been issued. By then, according to the 
Secretariat, some 42,000 positions had 
been evaluated by departments for 
eventual inclusion in the new General 
Services (GE) group. The Secretariat had 
anticipated that implementing the 
Standard across the public service would 
be spread over five years. During our 
audit, new versions of the Universal 
Classification Standard were issued and 
implementation was delayed several 
times. At the conclusion of our audit the 
Standard had not been submitted to 
Treasury Board ministers for approval. 
Exhibit 5.6 provides a schematic 
description of the Standard and its key 
characteristics as they appeared in the 
most recent version (June 1995) issued by 
Treasury Board Secretariat prior to the 
conclusion of the audit. 


Why Audit Before 
Implementation? 


5.23 We decided to conduct our audit 
before the new system was fully 
implemented because our experience with 
job classification and evaluation has 
shown that good design is essential to a 
new job evaluation plan: if significant 
problems in design are discovered after 
implementation, it is difficult to take 
effective corrective measures. Moreover, a 
number of concerns had been expressed 
by experts, unions and the Canadian 
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Human Rights Commission about the 
design and development of the Standard. 


5.24 Furthermore, several million 
dollars had already been spent by the 
Secretariat and departments to develop 
and test the Standard. Another reason for 
auditing before implementation was that, 
because classification and job evaluation 
are central to human resource 
management in the public service, 
implementing the Standard was likely to 
have a major impact on the payroll, labour 
relations, pay administration and many 
other human resource systems as well as 
efficiency of program delivery. 


3225 Audit objective. Our audit was 
aimed at assessing whether the Universal 
Classification Standard would be capable 
of evaluating all jobs in the public service 
(that is, universal) as well as ensuring that 
work performed by women would be 
made visible and valued in the same 
manner as men’s work to eliminate 
possible systemic bias (that is, 
gender-neutral). Another objective was to 
determine whether classification and job 
evaluation with the Standard and the 
occupational group structure would be 
simpler. Finally, we wanted to assess how 
the Treasury Board Secretariat had taken 
into account the cost/benefit factor. 


We decided to conduct 
our audit before the 
new system was fully 
implemented because 
our experience with 
job classification and 
evaluation has shown 
that good design is 
essential to a new job 
evaluation plan. 


5.26 Audit scope and approach. Our 
audit examined the way Treasury Board 
Secretariat managed the design, 
development, testing and validation of the 
new job evaluation plan and applied it to a 
number of positions using the June 1995 
or earlier versions of the Standard. We 
also looked at the work done for the task 
force or by the Secretariat to simplify the 
occupational group structure. We 
interviewed senior officials and human 
resource managers in a number of 
departments, and examined how other 
organizations in the public and the private 
sectors have simplified their occupational 
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Exhibit 5.6 


Schematic Description of the Universal Classification 
Standard and Its Key Characteristics 


Universal Classification Standard 


Its components: 
4 Factors 
16 Elements 
Degree Definitions 
Degree Illustrations 
Notes to Evaluators 


FACTOR CHRA* ———S«& ELEMENT MAXIMUM POINTS 


Service Delivery Effort Interaction 83 
Responsibility Influence 656 
Effort Thinking Challenge 480 
Effort Physical Demands 72 
Care and Responsibility Responsibility Work of Others 205 
Responsibility Care of Individuals 102 
Responsibility Financial Resources 152 
Responsibility Technical Resources ds 
Working Conditions Working Environment 76 
Conditions Risk to Health Tg 
Skills & Knowledge Skills Context 264 
Skills Acts & Regulations 163 
Skills Theories & Principles 534 
Skills Methods, Techniques & Practices 327 
Skills Communication V72 
Skills Physical Dexterity a2 
Total Points 3500 


*Relationship with the four factors specified in the Canadian Human Rights Act. 


Notes to Evaluators 


Specific instructions and guidelines on how each element is to be interpreted and applied 
and what characteristics of the work are or are not considered. 

Degree Definitions 
Each of the 16 evaluation elements has a number of degrees. These have been defined on 


a rating scale to represent a progression from a minimum to a maximum value. 


Degree Illustrations 


To assist evaluators in determining the appropriate degree of an element, each degree is 
exemplified by a number of degree illustrations. The degree illustrations have been 
selected from a variety of occupations to provide examples of the proper application of 
the degree. 


Source: Treasury Board Secretariat. The Universal Classification Standard (June 1995 version) 
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group structure and job evaluation 
systems. We also interviewed and 
consulted experts in compensation 
management. Details about the conduct of 
the audit are presented at the end of the 
chapter. 


5.27 When we began our audit, the 
government was committed to a policy of 
renewing the public service, providing 
flexibility for different needs but in the 
context of a single public service. While it 
can be argued that other organizational 
forms are possible — including 
decentralized human resource 
management — the audit was conducted 
against that policy. 


Observations and 
Recommendations 


The Status Quo Is Unacceptable 


5.28 There is a consensus in 
departments and among Treasury Board 
Secretariat officials that the current 
classification and job evaluation systems 
are both obsolete and too costly, given the 
current pressures for fiscal restraint, more 
efficient service delivery and downsizing. 
To them, the status quo is unacceptable. 
We concur with this assessment. 


5.29 We found, however, that while 
departments agree with the guiding 
principles identified by the task force for a 
new job evaluation system, they 
emphasize that any new system must 
allow for flexibility because of the 
significant variations in the nature and 
Operations of the programs and services 
they provide. Departments also believe 
that current and future budget reductions 
and ongoing restraint require that the 
design of the new system, its 
implementation and its ongoing 
management entail the lowest possible 
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cost in labour, time and dollars, as well as 
in human terms. 


Job Evaluation: A Significant 
Improvement but Not Yet Meeting 
the Mark 


The initial timetable for the project was 
unrealistic 


5.30 Given the context, and the 
technical difficulties associated with the 
project, Treasury Board Secretariat’s 
initial two-year timetable to develop a 
totally new job evaluation system as 
complex as the Standard was clearly too 
optimistic. Because of the initial deadline, 
the debate that should have taken place 
with numerous stakeholders on key issues 
did not take place. Issues conflicting with 
the deadline were ignored, although there 
were Clear signs that significant 
repercussions lay ahead. In particular, 
because of the tight deadline, the thorough 
testing necessary to assess the universality 
and gender neutrality of the Standard was 
not carried out. 


There is a consensus 
in departments and 
among Treasury Board 
Secretariat officials 
that the current 
classification and job 
evaluation systems are 
both obsolete and too 
costly. We concur with 
this assessment. 


Universality and gender neutrality were 
not assured 


eal | Treasury Board Secretariat took a 
number of measures to ensure universality 
and to reduce the risk of systemic bias 
against work predominantly performed by 
women, including: 


e the development of a single job 
evaluation plan that will make it easier to 
compare the value of jobs done by women 
and by men; 


e the incorporation into the Standard 
of the four factors set out in the Canadian 
Human Rights Act; 


e adefinition of gender neutrality that 
recognizes the need to identify and value 
properly the historical and traditional 
characteristics of the work of women and 
men; 


Report of the Auditor General of Canada — May 1996 


2 


The Reform of Classification and Job Evaluation in the Public Service 


In our opinion, the 
measures taken by the 
Secretariat were 
insufficient to 
conclude that the 
Standard could 
evaluate all jobs in the 
public service and ina 
gender-neutral 
manner. 


e the selection of 16 elements capable 
of covering a wider range of job content in 
the public service; and 


e extensive training of managers and 
human resource specialists on the new 
Standard and on the requirement for 
gender neutrality when drafting job 
descriptions and during the job evaluation 
process. 


5.32 The Treasury Board Secretariat 
concluded, in April 1995, that it was 
satisfied with the universality and gender 
neutrality of the Standard. In our opinion, 
however, the measures taken by the 
Secretariat were insufficient to conclude 
that the Standard could evaluate all jobs in 
the public service and in a gender-neutral 
manner. In fact, the following evidence 
suggests that the version of the Standard 
we reviewed was neither universal nor 
free of gender bias. 


The method of collecting information on 
job content needs to be strengthened 


5.33 If the Standard is to be capable of 
evaluating all jobs in the public service, 
comprehensive information needs to be 
gathered on the full range of significant 
job content present in the public service. 
However, we found serious deficiencies in 
the method that Treasury Board 
Secretariat initially used to collect job 
content information for the development 
of the Standard. The Secretariat collected 
job content information on a sample of 
1,000 positions throughout the public 
service. Although the sample seems large, 
the Secretariat acknowledged that it did 
not statistically represent the whole public 
service population. For example, it did not 
include all combinations of occupational 
groups, sub-groups and levels. Positions in 
predominantly female groups were also 
under-represented. 


5.34 Furthermore, the information 
gathered on job content was not subject to 


quality control to ensure that it was valid, 
reliable and gender-neutral. One pay 
equity expert consulted by Treasury Board 
Secretariat observed that the quality of the 
job content information gathered was 
inconsistent, varying from satisfactory and 
reasonable to incomplete and inaccurate. 
In addition, no effort was made before the 
first phase of the Standard’s 
implementation to verify whether the new 
job description format and content could 
collect information on all the significant 
aspects of the work performed by people 
in either predominantly male or 
predominantly female occupational 
groups. 


5.35 In spite of these deficiencies, the 
Secretariat used this job content 
information as a basis for making key 
decisions during the development of the 
Standard, particularly in choosing and 
defining the elements of the Standard that 
would serve to evaluate all jobs in the 
public service. This eventually led to work 
traditionally associated with women not 
being sufficiently visible, and to problems 
in applying the Standard to certain jobs. 


5.36 At the conclusion of our audit, 
the Treasury Board Secretariat had 
undertaken a thorough review of the 
literature to identify overlooked job 
content in predominantly female groups 
and was committed to ensuring that the 
Standard would make visible all 
significant job content. In addition, it was 
developing sampling techniques as well as 
quality control mechanisms, with the 
objective of ensuring universality and 
gender neutrality. 


The various components of the 
Standard had significant shortcomings 


Sor To assess the universality and 
gender neutrality of the Standard, one 
must assess whether its components 
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(elements, degree definitions and degree 
illustrations): 


e are clearly defined; 


¢ incorporate all significant content of 
all jobs in the public service; 


e include frequently overlooked 
aspects of work performed by women; 


e measure unique aspects of work 
performed, to avoid crediting the same job 
content more than once; 


e are consistent and appropriate within 
each level and for all levels; and 


¢ can properly value all significant job 
content found in the public service, 
including work performed by women. 


Exhibit 5.7 illustrates the various 
components of the Standard for an 
element that was entitled “Thinking 
Challenge” in the version we reviewed. 


5.38 Our assessment of each of these 
aspects using the June 1995 version of the 
Standard revealed that the Treasury Board 
Secretariat did not have the basis for 
asserting that the Standard was universal 
and gender-neutral. In particular, the 
version of the Standard we reviewed 
would, to a large extent, perpetuate gender 
bias in job evaluation. Unless corrective 
measures are taken, this could lead to 
numerous costly and time-consuming 
grievances or complaints under the 
Canadian Human Rights Act and cast 
doubt on the Standard’s credibility as a 
universal and gender-neutral job 
evaluation plan. We noted the following 
deficiencies: 


¢ Except for a few elements, 
definitions were vague, broad and at times 
ambiguous. It was not clear what was 
actually being measured. 


¢ Degree definitions aimed at 
providing, for each element, a rating scale 
representing a progression from a 
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minimum to a maximum value were also 
extremely vague and ambiguous. 


e Degree illustrations intended to 
exemplify, for a specific element, degrees 
in concrete job content and to show the 
progression from one level to another 
contained the most significant 
deficiencies. In many cases, the criteria 
embedded in them were different from 
those in the element definition or in the 
degree they were supposed to illustrate. 
Aspects of women’s work including work 
at a high level were missing. Also, 
progression was unclear because 
consistency was a problem in many cases. 
(The Appendix provides a summary of the 
significant deficiencies found in the 


various components of the Standard.) At the conclusion of 


our audit the Treasury 
Board Secretariat was 
taking a number of 

important measures to 
address our concerns. 


e Statistical tests to diagnose for 
systemic effects by gender revealed that 
six of the 16 elements of the Standard 
measured male and female job content 
equally well. The 10 remaining elements 
favoured the content of predominantly 
male jobs. Such results are indicators of 
potential bias. 


5.39 At the conclusion of our audit the 
Treasury Board Secretariat was taking a 
number of important measures to address 
our concerns, such as: redefining, 
renaming and regrouping the elements; 
identifying gaps and examining the need 
for new elements; developing matrices 
and grids to discipline progression among 
degrees; and developing a framework to 
guide the review and the rewriting of 
degree illustrations. 


The weighting of the Standard’s 
elements needs to be reassessed 


5.40 All job evaluation plans deem 
some job content to be more important 
than other content. The weighting — that 
is, the percentage of points awarded to 
each element in a job evaluation system 
— is intended to establish the relative 
importance of each element, reflecting the 
organization’s objectives and values. The 
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Exhibit 5.7 


Illustration of the Various Components 
of the Element “Thinking Challenge” 


Element Definition 


See oy This element measures the effort of the thinking challenge in terms of the complexity of the 
Thinking Challenge demands and outcomes associated with the work performed. 
Notes to Evaluators 
Notes to Evaluat OK In assessing this element, evaluators are to consider the variety and difficulty of the decisions 


to be made, problems to be solved, options to be considered and choices to be made. 


Degree 1 Limited Effort: 
J Demands and outcomes are of limited complexity. 
Degree Definitions _ Degree 2 Some Effort: 
/ Demands and outcomes are somewhat complex. 
Degree 3 Moderate Effort: 
Demands and outcomes are moderately complex. 
Degree 4 Significant Effort: 


Demands and outcomes are significantly complex. 


Degree 1 Limited Effort: 


The work involves operating a photocopying machine and sequencing jobs in the order 
they are received, processing incoming facsimile messages and calculating monthly 
telephone charges. 


Degree2 Some Effort: 


The work involves assessing the situation and selecting and operating light and heavy 
equipment to maintain roads or runways in usable condition in response to weather 


Degree Illustrations Sonditions: 


Degree3 Moderate Effort: 


The work involves developing and adapting training plans and materials, evaluating 
individual students’ training needs and providing additional instruction or remedial work 
as problems are detected. 


Degree 4 Significant Effort: 


The work involves evaluating seed and grain qualities, analyzing the quality control 
process of fertilizer and pesticide manufacturers and the quality of inspection training 
programs, assessing mills and plants against standards to detect anomalies, analyzing 
complaints and reviewing corrective actions for compliance. 


Source: Treasury Board Secretariat. The Universal Classification Standard (June 1995 version) 
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values themselves must be free of gender 
bias. Exhibit 5.6 shows the maximum 
possible number of points allocated to 
each of the 16 elements of the Standard. 


5.41 To determine the importance of 
each element, 300 positions drawn from a 
sample were ranked using a method 
known as “whole-job-ranking”. The 
method consists of comparing jobs using 
job descriptions, to rank them on the basis 
of their overall global “value”. 


5.42 Experts recognize that the 
“whole-job-ranking” method is fraught 
with difficulties for ensuring gender 
neutrality and validity, especially when 
applied to a large number of positions. 
Because it is subjective, the ranking can 
be influenced by a number of irrelevant 
factors, such as stereotyping. Evaluators 
are unlikely to question their preconceived 
opinions of the overall importance of the 
work, including their possible gender bias. 
If such a method is to be used, a 
significant number of safeguards must be 
in place to prevent or to detect bias 
potential. We found that the measures - 
taken by the Secretariat were insufficient. 


5.43 Although the results of the 
ranking exercise showed that the positions 
of some predominantly female groups 
moved up in the new hierarchy, our 
review of Treasury Board Secretariat’s 
statistical analysis of these results 
nevertheless suggested the presence of 
significant problems. For example: 


e aspects of women’s work were 
valued unfavourably in the ranking 
CXercise; 


¢ implicit criteria used to evaluate jobs 
were very limited: in fact, a few aspects of 
job content alone could have served to 
predict the ranking of all 300 jobs; and 
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e there were indications that the same 
aspect of work was credited more than 
once. 


5.44 In spite of evidence of significant 
deficiencies, the ranking results were used 
to develop the weighting of the elements 
contained in the version of the Standard 
we reviewed. We found that the resulting 
weighting was highly imbalanced. Four of 
the 16 elements were assigned more than 
55 percent of the points; 10 elements were 
each assigned a value of 5 percent or less, 
and are therefore given negligible weight. 
In the version of the Standard we 
reviewed, the only element designed 
explicitly to take into account work 
traditionally associated with women, 
“Care of Individuals,” received a value of 
approximately 3 percent. 


5.45 Overall, we found that, in the 
version of the Standard we reviewed, the 
importance given to three elements, 
“Influence”, “Theories and Principles” 
and “Thinking Challenge” — often 
associated with administrative/managerial 
work — was such that the other 

13 elements would not likely affect job 
evaluation results to any great degree. 


5.46 The imbalance in the weight of 
the elements limits the Standard’s ability 
to value all jobs in the public service 
properly. Furthermore, it results in the 
undervaluation of work performed by 
women. The high concentration of 
weights in only a few elements, the large 
number of elements to which no 
appreciable value is attributed and the low 
value assigned to work traditionally 
performed by women are due to 
shortcomings in the “whole-job-ranking” 
exercise noted earlier. 


5.47 At the conclusion of our audit, 
the Treasury Board Secretariat was 


We found that the 
importance given to 
three elements, 
“Influence”, “Theories 
and Principles” and 
“Thinking Challenge” 
— often associated 
with administrative/ 
managerial work — 
was such that the 
other 13 elements 
would not likely affect 
job evaluation results 
to any great degree. 
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The Treasury Board 
Secretariat did not 
conduct a full-scale 
pilot project to test the 
universality and 
validity of the 
Standard. 


planning to examine the weighting of the 
elements of the Standard. 


Thorough testing of universality and 
gender neutrality is essential 


5.48 Because of the self-imposed tight 
schedule, the Treasury Board Secretariat 
did not conduct a full-scale pilot project to 
test the universality and validity of the 
Standard. Instead, it conducted three 
limited tests at different phases of the 
Standard’s development. In our opinion, 
this testing was not enough to assert that 
the Standard can evaluate all positions in 
the public service. 


5.49 For example, during the 
development phase, a sample of about 
800 positions was used to assess the 
completeness and applicability of the 
Standard. Statistical tests for universality 
conducted by the Treasury Board 
Secretariat indicated problems in applying 
the Standard to particular occupational 
groups, such as scientific personnel or 
professionals. No analysis could be done 
for positions in many other occupational 
sub-groups because samples were too 
small. Nor could statistical tests be 
conducted for gender neutrality because 
the sample of positions from 
predominantly female groups was 
inadequate. 


5.50 A second test began with the 
application of the Standard to some 
35,500 positions (about 25 percent) of the 
125,000 positions to be included in the 
new General Services group (GE). The 
objective was to verify whether the weight 
given to each element was appropriate and 
to determine the number of levels this 
occupational group could have. Although 
this was to be a test of the Standard, the 
Treasury Board Secretariat did not 
consider it appropriate or necessary to 
establish adequate quality control 
mechanisms to ensure that all departments 


were conducting the test properly. We 
found significant deficiencies in this test. 
For instance: 


e No guidelines were issued to ensure 
that departments would use a valid and the 
same sampling method. 


e While testing was being carried out, 
the Secretariat made the weighting 
scheme available to departments upon 
request. Usually, the weights for the 
elements of a new job evaluation system 
are not disclosed at that stage to ensure 
that raters are not unduly influenced by 
the rating points instead of by the job 
content. 


e “Jobbing” was used by many 
departments. “Jobbing” means describing 
and evaluating one position and 
automatically giving the same value to 
similar positions without necessarily 
evaluating them individually. While 
perfectly acceptable and even desirable at 
the implementation stage, “jobbing” is not 
advisable for testing purposes because it 
can seriously distort overall results. 


e Although the Treasury Board 
Secretariat reported at the end of this test 
that some 35,500 positions had been 
evaluated, we found that Secretariat 
officials did not know how many and 
which individual positions had really been 
evaluated. Nor did they know the results 
of the evaluation for the specific positions 
actually evaluated. One department, using 
the jobbing technique, reported 6,900 
evaluation results when in fact only 
30 positions had been evaluated. 


5.51 Because of these deficiencies, the 
test results cannot be used to assert the 
validity and universality of the Standard 
for positions to be included in the new 
General Services (GE) group. 


5.52 In July 1994, in response to our 
initial concerns about universality, 
Treasury Board Secretariat conducted 
additional testing on positions in 
occupational groups other than the 
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General Services group. We found that 
some 70 occupational groups or 
sub-groups were excluded from the 
sample for this test — including some that 
were predominantly female. In a number 
of cases, the sample also included too few 
positions to be able to determine patterns, 
if any. 


55 Our examination of the results of 
the various tests conducted by the 
Secretariat indicates that: 


* positions at the same level under the 
existing job evaluation system were 
distributed over three, four and sometimes 
five different levels in the new proposed 
General Services group; 


* positions that had been given about 
the same number of points using the 
Standard would have salary differences 
ranging from $5,000 to $50,000 using the 
current salary scales; and 


e anumber of positions were 
exceeding the maximum number of points 
proposed for the highest level of the 
General Services group. 


While such results might be justified and 
valid given the new values imbedded in 
the Standard, they might also indicate the 
presence of problems or anomalies. A 
detailed examination of the results is 
necessary to determine if they are valid or 
stem from shortcomings in the Standard’s 
components, misapplication of the 
Standard, skewed results due to “jobbing”, 
improper classification of jobs under the 
current system or the way pay rates have 
been established. The Secretariat has yet 
to determine if such results are valid or 
not. 


5.54 We believe that the Secretariat 
needs to conduct more rigorous and 
convincing testing to assert the validity, 
universality and gender neutrality of the 
Standard if it wants to persuade the 
employees, unions and other stakeholders 
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that the Standard is indeed valid, universal 
and gender-neutral. 


3.05 Treasury Board Secretariat, in 
collaboration with departments and 
representatives of employees and with 
the support of external experts in pay 
equity and job evaluation, should ensure 
that the Standard is capable of 
evaluating all jobs in the public service 
while meeting criteria for gender 
neutrality stemming from the Canadian 
Human Rights Act. More specifically, 
the Secretariat should ensure that: 


e the method used for collecting 
information on job content in public . 
service jobs is valid, reliable, credible We believe that the 
and free of bias, with appropriate Secretariat needs to 


lit trol hani t i : 
i Te ay apa mn Sota conduct more rigorous 
the collection of information on all 


significant job content, including work and convincing testing 
associated with women’s work that is to assert the vali dity, 


often overlooked; 
* the sample of jobs used to gather universality and 
information on job content is adequate gender neutrality of 


and credible, with adequate gender the Standard 
representation; _ ; 


¢ the various components of the 
Standard (elements, degrees and degree 
illustrations) are reviewed to ensure 
that they can effectively capture and 
properly value all the significant job 
content found in the public service, 
including that associated with women’s 
work. This should include ensuring 
that: 


— all components of the Standard 
are clearly defined and measure 
unique aspects of work to the 
extent possible; 


— the progression from minimum 
to maximum value in each element 
is clear; and 


— illustrations intended to assist 
raters provide a reasonable 
spectrum of job content, do not 
undervalue work performed by 
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One of the reasons for 
the continuing high 
cost is that, with the 
Standard, job content 
information is still 
gathered through job 
descriptions. 


women and are consistent with the 
definitions and the desired 
progression in value; 


e the weight allocated to each 
element is appropriately distributed to 
permit the valuation of all jobs, 
including work performed by women, 
on a fair and equitable basis while 
giving proper recognition to the 
objectives and values adopted for the 
public service and the reform; and 


e thorough testing, on a valid 
sample, of the validity, reliability and 
gender neutrality of the Standard is 
carried out with adequate quality 
control measures and, if necessary, 
review and additional testing before 
implementation. 


Treasury Board Secretariat’s response: 
Treasury Board Secretariat (TBS) agrees 
with these recommendations. 


TBS acknowledges that the version of the 
Universal Classification Standard (UCS) 
that was reviewed by the Auditor General 
had shortcomings. However, a lot was 
accomplished and a leading-edge tool was 
produced at that time. TBS agrees that 
some things could have been done 
differently given more resources and better 
co-operation among the stakeholders. 


TBS has since adopted a consultative 
approach to develop improvements to the 
UCS, enhance the collection of data, 
re-weight elements and add new elements, 
and simplify the process. All of the main 
stakeholders have recently agreed upon 
definitions for gender neutrality, 
universality and simplification. 


The current active participation of the 
Canadian Human Rights Commission, the 
Auditor General’s Office, consultants, 
departments and unions on a UCS 
Advisory Committee, a Technical 
Committee, and other committees and 
working groups dedicated to specific 
aspects of the improved UCS should 
ensure that we can address previous 


shortcomings and succeed in achieving 
these recommendations. 


Statistical testing will also supplement the 
qualitative approaches being developed in 
consultation with all stakeholders to 
achieve a universal, gender-neutral plan. 


Simplification and Operating 
Costs of Job Evaluation 


Job evaluation will be improved but still 
time-consuming and costly 


5.56 
standard will make it much easier for 
managers to make classification decisions 
and to explain the results. Still, job 
evaluation with the version of the 
Standard we reviewed would be relatively 
complex and costly, according to experts. 


EY) One of the reasons for the 
continuing high cost is that, with the 
Standard, job content information is still 
gathered through job descriptions. Job 
descriptions in the new format we 


Using only one classification 


reviewed were long, in many cases 
ranging from five to 11 pages. The 
Treasury Board Secretariat estimated that 
it would take approximately two days to 
write a job description using the new 
format. Departments we visited generally 
confirmed this estimate, some indicating 
that, in many cases, it took as long as 
three days. The time and thus the cost of 
writing job descriptions is about the same 
with the Standard as with the present job 
evaluation plans used by departments. 
However, Treasury Board Secretariat 
estimates that with the Standard, a 
committee will be able to evaluate 

five positions a day on average, a notable 
improvement compared with the 

one position a day using the existing 
systems. Some departments we visited, 
however, thought that this estimate was 
optimistic. 


5.58 Writing detailed job descriptions 
and evaluating jobs with the version of the 
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_ Standard we reviewed entail significant 
training costs. It was estimated that 
individuals need two or three days of 
training to be qualified in writing job 
descriptions in the new format, and 
another two or three days to be able to 
participate in job evaluation committees. 
One department reported that the cost of 
training managers to write job descriptions 
and evaluate jobs with the Standard had 
been $3.4 million in salary alone, for 
some 18,000 positions. On this basis, the 
cost in salary for training managers, let 
alone incumbents of positions, in the 
public service as a whole could be 
substantial. 


309 We found that some departments 
are exploring avenues to reduce the 
burden and operating costs of job 
evaluation after the Standard is 
implemented. For instance, one 
department with some 25,000 positions 
intends to use the “jobbing” technique 
extensively. It has identified 82 generic 
job descriptions that could be applied to 
about 65 percent of all its positions 
proposed for inclusion in the General 
Services (GE) group. This would reduce 
significantly the time required to evaluate 
them. Other departments have developed 
or are planning to introduce computerized 
job descriptions or a computer bank of job 
descriptions. 


Effort to computerize job evaluation 
failed 


5.60 In the context of a suggestion 
made by the task force, the Treasury 
Board Secretariat launched in April 1991 
a research and development project to 
determine whether artificial intelligence 
technology could provide managers with a 
tool for writing job descriptions and for 
evaluating jobs in a few minutes. 


5.61 With co-operation and financing 
from 10 departments and with assistance 
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from the Artificial Intelligence Research 
and Development Fund of Industry, 
Science and Technology Canada, the 
Secretariat hired two consultants to 
produce a prototype with two modules — 
one for writing job descriptions and one 
for evaluating jobs. In March 1993, the 
prototype was delivered. 


5.62 Consulting and Audit Canada 
was asked by the Secretariat to assess the 
results. The report revealed that, in spite 
of impressive achievements in research, 
there were a number of serious problems 
with the design and the different 
technology used for the two modules. As a 
result, the prototype could not be made 
operational, and the effort required to 
develop an operational system would 
mean almost starting over. The prototype 
would have to be redesigned. Consulting 
and Audit Canada estimated that, to 
produce an operational computerized tool, 
work would have to continue for another 
two to three years and could cost 

$3.5 million to $6 million. Furthermore, 
Consulting and Audit Canada considered 
that the project would involve significant 
technical risk. For these reasons it 
recommended a cost/benefit analysis 
before pursuing the project. After 

$1 million had been spent, the project was 
terminated. Departments that had 
contributed resources to the project 
expecting an operational system were 
disappointed by the results. 


5.63 We believe there are ample 
opportunities to simplify job evaluation 
further and to reduce the associated costs. 
Between a job evaluation system that 
requires detailed and lengthy job 
descriptions and the individual evaluation 
of jobs and a system that does the same 
thing using artificial intelligence 
technology, there are intermediate 
solutions that can take advantage of 
current computer technology, and new 


We found that some 
departments are 
exploring avenues to 
reduce the burden and 
operating costs of job 
evaluation after the 
Standard is 
implemented. 


We believe there are 
ample opportunities to 
simplify job evaluation 
further and to reduce 
the associated costs. 
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The simplification of 
job evaluation and, in 
particular, reduction of 
the operating costs is 
an important issue for 


trends in job evaluation such as 
developing a structured computerized 
questionnaire. Exhibit 5.8 presents new 
trends in job evaluation. Exhibit 5.9 
presents an illustration of the structured 
questionnaire approach to job evaluation 
for “Work of Others”. The simplification 
of job evaluation and, in particular, 
reduction of the operating costs is an 
important issue for departments. They will 
bear most of the implementation cost, as 


5.64 
the Treasury Board Secretariat was 
examining various options to improve the 
collection of information on jobs at lower 
cost, including the feasibility of using a 
questionnaire during the implementation 
phase. Other options being considered 
include data banks of job descriptions and 
job evaluations, and computerizing a 
questionnaire at a later stage. 


At the conclusion of our audit, 


departments. well as the ongoing operating costs 5.65 Treasury Board Secretariat, in 
resulting from any job evaluation plan. collaboration with departments and 
Their demands for keeping such costs as representatives of employees and with 
low as possible should not be ignored. the support of external experts in pay 
Exhibit 5.8 


New Trends in Job Evaluation: Simplification, 
Computerization and Reducing Operating Costs 


Our review of recent literature on job evaluation and visits and consultations with private and public sector organizations showed 
that organizations are moving away from using job descriptions in evaluating jobs. One emerging approach is the use of a 
structured questionnaire as input to a computerized job evaluation system. Many organizations we visited that had set objectives 
of simplifying job evaluation and reducing operating costs had developed such questionnaires. 


It is clear from both the literature and the results of our visits and consultations that developing a well-designed, structured, 
computerized questionnaire is very demanding, and that it requires a greater initial investment than writing job descriptions. 
However, organizations that opted for a questionnaire consider that the advantages significantly offset both the difficulty and the 
initial cost. The reported advantages of structured computerized questionnaires include the following: 


¢ They have the support of pay equity experts and organizations responsible for enforcing pay equity prin- 
ciples: a well-designed, structured questionnaire can ensure that all the job content of female- and male- 
dominated groups is made visible and valued properly. 


¢ The collection of job content information and the evaluation of that job content can be done simultaneously. 


¢ The time required to administer the questionnaire varied from 90 minutes to 3 hours, compared with two or 
three days for writing job descriptions. 


* The demands on a job evaluation committee are significantly reduced, except for the conversion to the new 


system. A committee is still needed, however, to validate the results and oversee the management of the sys- 
tem afterward. 


e Training is reduced significantly, since it is focussed mainly on the needs of classification specialists. 


Partly because the design of the questionnaires eliminates the problem of inconsistency in the language used to describe the work, 
they can significantly minimize the inflation of evaluation ratings. In many of the organizations we visited, this reduction in the 


scope for manipulating the system resulted in major drops in the volume of requests for reclassification and in the number of staff 
assigned to manage the job evaluation system. 


A questionnaire does not eliminate the need to audit and monitor the classification decisions made while a new job evaluation 


system is being implemented, or over the years. However, the time devoted to these activities is reduced significantly because of 
controls built into the questionnaire or the computer program. 


Those organizations we visited emphasized, however, that effective communication — including extensive information sessions 
—— was essential to ensure transparency and to make sure that managers and employees understood the new job evaluation plan. 


5-24 Report of the Auditor General of Canada — May 1996 


equity and job evaluation, should ensure 
that the implementation costs and 
future operating costs borne by 
departments are the lowest possible, 
notably by exploring alternatives to 
detailed job descriptions, such as a 
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Treasury Board Secretariat agrees. 


of the Standard to ensure that work 
characteristics are not overlooked. 


structured questionnaire, and by finding of writing and evaluating work 


ways to simplify the job evaluation 


descriptions are being addressed in 


process further. 


Treasury Board Secretariat’s response: 


However, it is important to separate out 
investment that goes into the design phase 


Streamlined and more economical means 


Exhibit 5.9 


Illustration of a Structured Questionnaire Approach 
to Job Evaluation: One Question for the Element 
“Work of Others” 


This section asks questions about your job’s responsibility for the work of others. “Others” could be subordinates, secondees, 
project team members, volunteers, agency personnel and students working on a temporary or short-term basis — but does not refer 


Work of Others 


to work performed by individuals under personal or professional service (which is measured elsewhere). 
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-h 
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Circle the Best Answer. 


Not part of the job 
Small part of the job 


BRWN eR 


Important part of the job 


How important to your job is: 


Making up a duty roster or work schedule for others 

Seeing that the duty roster or schedule is followed 

Making changes to the roster or schedule when priorities change 
Assigning tasks or work during a shift or project 

Arranging for others’ on-the-job training 


Checking the work of others 
Initiating performance appraisals 
Giving input on performance appraisals 


Final review of performance appraisals 
Recommending or initiating disciplinary actions 
Carrying out disciplinary actions 

Authorizing disciplinary actions 

Counselling others about their performance 


Recommending new or changed work procedures 
Implementing new or changed work procedures 
Authorizing new or changed work procedures 
Establishing standards for work 


Developing priorities for the work of others 

Approving priority plans 

Developing long-term objectives for work (staffing, resource requirements) 
Approving long-term objectives for work 


Recommending training for others 
Authorizing training for others 


Somewhat important part of job 


— SS eS 
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With a few exceptions, 
all the requests made 
to both the task force 
and Treasury Board 
Secretariat were aimed 
at maintaining current 
occupational groups 
or creating sub-groups 
within the 
occupational group 
structure. 


Some departments 
have made 
representations to the 
Treasury Board 
Secretariat stressing 
the importance of 
simplifying the 
occupational group 
structure further. 


anticipation of a positive decision on 
implementation. 


Simplification of the Occupational 
Group Structure 


5.66 As we note in paragraph 5.17, the 
task force set up to review the design of 
the occupational structure recognized its 
complexity and recommended reducing 
the 72 occupational groups and 

106 sub-groups to 23 and 8 respectively. It 
also recommended the creation of the 
General Services group that would include 
some 125,000 employees. The rationale 
for whether to merge or maintain 
occupational groups was based mainly on 
the general nature of the work and the 
community of interests. Other factors 
were also considered, such as the need for 
professional certification, comparability 
with outside labour markets and respect 
for union affiliations. 


The occupational group structure: less 
simplified than recommended 


5.67 The Treasury Board Secretariat 
endorsed the criteria used by the task 
force for the simplification of the 
occupational group structure but, under 
pressure, placed greater importance on its 
commitment to respect union affiliation. A 
review of representations and 
correspondence revealed the extent of the 
lobbying efforts on this point by groups, 
associations and unions. With a few 
exceptions, all the requests made to both 
the task force and Treasury Board 
Secretariat were aimed at maintaining 
current occupational groups or creating 
sub-groups within the occupational group 
structure. As a result, instead of the 

8 sub-groups recommended by the task 
force there could be as many as 30 to 

45 sub-groups. In that case, simplification 
would be much less than initially 
envisaged. 


The proposed structure: varying 
degrees of support 


5.68 The proposed new occupational 
group structure does not appear to have 
the support of all departments. For 
example, in February 1992, deputy 
ministers reaffirmed their support for the 
objectives of the reform but they 
expressed concern about the large number 
of sub-groups and the apparent lack of 
consultation with departments on 
decisions about the types and number of 
occupational groups. Many felt that 
respect for union affiliations had taken 
precedence over departmental priorities 
and operational requirements. 


5.69 Most of the departments we 
visited were satisfied with the creation of 
the General Services group, especially 
those with a large proportion of employees 
proposed for inclusion in it. These 
departments saw the creation of the group 
as a major step in the right direction. They 
also felt that this new occupational group 
would make it easier to deal with 
expenditure reductions, restraint and 
restructuring. 


5.70 Departments less affected by the 
creation of the General Services group 
said that the number of occupational 
groups and especially sub-groups was still 
too high. They saw the proposed 
occupational group structure as only a 
marginal improvement. For example, 
some would like to have seen a number of 
occupational groups in the department 
amalgamated into a single, broader group 
to facilitate career advancement, 
maximize flexibility or meet operational 
requirements more efficiently. Some 
departments have made representations to 
the Treasury Board Secretariat stressing 
the importance of simplifying the 
occupational group structure further. 


S71 Departments also expressed 
concern about the appropriateness of 
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maintaining current union accreditations 
in the transition period. For many 
departments, this runs the risk of 
significantly diluting the anticipated 
benefits of simplification. Experts also 
said that maintaining a large number of 
occupational groups and the current 
collective bargaining structure, where 
each bargaining agent representing an 
occupational group negotiates salaries 
separately, will likely conflict with the 
principle of equal pay for work of equal 
value and the use of internal relativity as 
the underlying principle of compensation 
in the public service. In their opinion, 
maintaining the current bargaining 
structure and process for negotiating 
salaries could eventually have a negative 
impact on any improvement in internal 
equity brought about by the application of 
the Standard. 


Significant opportunities exist to 
simplify further 


5.72 Our review of the work and 
studies carried out in the context of the 
task force indicated that other options 
would have permitted greater 
simplification of the occupational group 
structure and might have offered more 
flexibility. There is no evidence that such 
options were considered fully. For 
example, one option advocated retaining 
six occupational groupings to replace the 
current 72 occupational groups. That 
option also assumed that all jobs would be 
evaluated using a single job evaluation 
plan. Its proponents claimed that it would 
eliminate barriers among the existing 
groups and would provide much greater 
flexibility. 


5.13 We also found that other 
organizations — including public sector 
organizations — have undertaken more 
vigorous reforms of their occupational 
structure than those envisaged by the task 
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force. For instance, one organization has 
reduced its occupational groups from 27 
to one. Instead of some 260 pay rates it 
now has a 16-level pay structure, ranging 
from cleaner to vice-president. According 
to senior management, this new structure 
provides simplified pay administration, 
increased mobility and management 
flexibility. 


5.74 As part of its reforms, the 
Australian public service has also 
simplified its occupational group 
structure. It now has only five broad 
groupings, including an equivalent of the 
General Services group, the 
Administrative Service Officer. 


3015 Because of the close links 
between the occupational group structure 
and other aspects of human resource 
management in the public service, such as 
staffing, pay and labour relations, further 
simplification of the occupational 
structure and related administrative 
systems could result in substantial savings, 
while improving efficiency. For instance, 
an analysis of somé 630 pay scales in the 
public service (regional rates were 
excluded) revealed that often differences 
among them were not significant. When 
we applied a differential of 4 percent and 
compared the highest rates of these pay 
scales, we found that there were fewer 
than 50 significantly different pay scales 
in the public service. 


5.76 For all these reasons, we believe 
that more could be done to either simplify 
the occupational group structure and/or 
reduce the administrative cost of other 
systems that results from maintaining a 
complex occupational structure, such as 
the current one or even the one proposed. 


Further simplification 
of the occupational 
structure and related 
administrative 
systems could result 
in substantial savings. 


baa [7 Treasury Board Secretariat, in 
collaboration with unions, departments 
and other stakeholders as appropriate, 
should review the proposed 
occupational group structure to 
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Most departments 
anticipated that payroll 
costs would increase 
as a result of 
converting to the new 
system, though they 
did not know by how 
much. 


Departments need to 
know the cost 
implications as well as 
the benefits they will 
reap from the reform. 


determine how it could be simplified 
further and/or how its impact on the 
complexity and cost of related human 
resource systems such as collective 
bargaining, staffing, pay administration 
and others could be significantly 
reduced. 


Cost Neutrality: Not Clearly 
Defined and Possibly Unrealistic 


5.78 At the beginning of the reform 
project, Treasury Board Secretariat made 
a commitment that the reform would be 
“cost-neutral’”’. We found that the term had 
not been defined clearly and that, as a 
result, there were several interpretations of 
its meaning. Furthermore, we found that 
the cost of operating the existing job 
evaluation systems was not known with 
any degree of precision. Treasury Board 
Secretariat had taken steps to determine 
the cost of the current systems and the 
potential savings stemming from the 
implementation of the new reformed 
system, but without success. Scenarios 
and cost estimates we reviewed ranged 
from $65 million to $200 million a year 
for some 200,000 classification actions. 
Partly because of this, at the conclusion of 
our audit in the fall of 1995, the 
Secretariat was not in a position to 
determine with any degree of precision the 
costs and benefits of the new system. 


SEAL Departments’ officials did not 
know how the objective of cost neutrality 
could be achieved, given the requirements 
for pay equity and the guarantee contained 
in current collective agreements that any 
employee whose position is evaluated to a 
lower level will not see a reduction in 
salary. Most departments anticipated that 
payroll costs would increase as a result of 
converting to the new system, though they 
did not know by how much. Some 
expressed concerns that they may incur 
higher cost without much significant 


benefit, because the impact of the new 
General Services (GE) group on the 
efficiency of their operations would be 
more limited than in other departments. In 
times of restraint, return on investment is 
an important issue for departments. One 
department reported having invested some 
$17 million on the first phase of applying 
the Standard to approximately 

18,000 positions. This figure excluded the 
cost of converting to the new system. We 
must recognize, however, that departments 
whose programs are offered on a 
standardized basis may have incurred 
much lower costs up to now. Departments 
need to know the cost implications as well 
as the benefits they will reap from the 
reform. 


Treasury Board Secretariat’s comments: 
Treasury Board Secretariat asserts that 
costs during the development of a 
Standard, when the fullest range of 
information is being sought and the 
training curve is at its steepest, will be 
higher than those incurred during the 
actual implementation of the Standard. 
Therefore, these costs should not be 
projected as being the likely or typical 
departmental costs of implementing the 
Standard. 


Treasury Board Secretariat agrees that 
costs can be reduced by simplified 
methods during implementation and, to 
this end, is pursuing a variety of 
mechanisms including computer-based 
questionnaires, generic work descriptions, 
and cost-effective training modules. 


5.80 Two years following the tabling 
of this Report the Treasury Board 
Secretariat should conduct an 
evaluation of the progress of the reform. 
In conducting its evaluation the 
Secretariat should take into account the 
views of the various stakeholders and 
the impact of the reform on them. 


Treasury Board Secretariat’s response: 
Treasury Board Secretariat agrees in 
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principle with this recommendation and its 
scope, and will raise the matter at an 
early meeting of the UCS Advisory 
Committee with a recommendation that it 
be endorsed. 


Need for a Different Approach 


Treasury Board Secretariat’s initial 
positions contributed to problems 


5.81 We feel that some initial 
positions adopted by the Treasury Board 
Secretariat contributed to an impasse with 
some stakeholders and experts and were 
partly responsible for many of the 
problems we identified. For example, staff 
in the Secretariat took the position 
throughout the audit that as long as male 
and female work was measured by the 
same standard, the evaluation of the work 
was gender-neutral. Such a view supports 
systemic discrimination: one must ensure 
that all aspects of men’s and women’s 
work are reflected in the components of 
the Standard. 


5.82 We also found that until recently, 
the Secretariat did not give sufficient 
consideration to advice it received. For 
instance, in April 1991 a pay equity expert 
informed the Secretariat that the proposed 
methodology for developing the weighting 
of the Standard’s elements was flawed and 
possibly did not meet the requirements of 
section 11 of the Canadian Human Rights 
Act. We have no evidence that the 
Secretariat examined this serious 
allegation thoroughly when it was raised. 
As we have noted in paragraph 5.46, this 
methodology was one of the causes of the 
significant imbalances in the weights 
given to the elements of the Standard. 


5.83 The Secretariat’s initial 
interpretation of the exclusive and 
unilateral right of Treasury Board to 
develop a classification and job evaluation 
plan also led to problems. It is true that 


The Reform of Classification and Job Evaluation in the Public Service 


the Public Service Staff Relations Act 
explicitly excludes job classification or 
job evaluation from collective bargaining, 
and provides for no consultative or 
negotiation framework for the meaningful 
joint work required to develop a valid and 
credible classification and job evaluation 
plan. However, we think that the 
complaint mechanism provided in the 
Canadian Human Rights Act of 1978 
requires a change in attitude and approach 
on the part of the employer, as well as 
some form of framework to discuss issues 
related to job evaluation. Employees or 
their representatives can challenge a job 
evaluation system developed by the 
employer, including some of its 
underlying premises, if they think that it 
discriminates on the basis of sex. If 
challenged, the employer must defend its 
decisions before an independent third 
party. Also, empirical evidence and 
jurisprudence point to the fact that 
systems developed in collaboration with 
employees are more likely to be balanced 
and representative of the values of all 
employees, and less discriminatory. 


We feel that some 
initial positions 
adopted by the 
Treasury Board 
Secretariat contributed 
to an impasse with 
some stakeholders 
and experts. 


Until recently, lack of sustained 
consultation with stakeholders and 
experts 


5.84 We noted a lack of sustained 
consultation, in particular with certain 
stakeholders such as the Canadian Human 
Rights Commission, external pay equity 
or job evaluation experts and unions, 
throughout the Standard’s development. 
Although the Secretariat conducted 
numerous consultations with these 
stakeholders, they often were on very 
specific topics, were conducted separately 
and lacked continuity. No fundamental 
issues were debated with all the 
stakeholders or experts present, such as 
the best method to use to determine the 
weighting of the elements of the Standard. 
The lack of sustained consultation resulted 
in increasing distrust of the Secretariat’s 
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The lack of sustained 
consultation resulted 
in increasing distrust 
of the Secretariat’s 
approach. 


approach. Some departments had similar 
problems with consultations held by the 
Secretariat. They perceived these 
consultations as intended more to convey 
or confirm decisions than to really solicit 
points of view or analyze and discuss 
options for solutions before decisions were 
made. 


5.85 At the conclusion of our audit, 
the Treasury Board Secretariat was putting 
in place a number of committees aimed at 
bringing stakeholders and experts 
together. 


Conclusion 


The Challenge Only Partially Met 


5.86 Almost six years have passed 
since the task force on classification and 
job evaluation made its recommendations 
to simplify the occupational group 
structure and job evaluation in the public 
service. Although the development of the 
Universal Classification Standard is, itself, 
a remarkable achievement in many ways, 
given our findings we have to conclude 
that the challenge posed by the reform has 
been only partially met. 


Significant Issues Need to Be 
Addressed 


5.87 While some significant progress 
has been made, substantial work must be 
done, notably to ensure that the Universal 
Classification Standard is a valid and 
reliable tool capable of evaluating all jobs 
in the public service. Similarly, greater 
efforts have to be made to ensure that 
work performed by women and 
traditionally overlooked is made visible 
and valued appropriately and in the same 
manner as work performed by men. If the 
version of the Standard we have reviewed 
had been implemented, it would not have 


met the requirements for pay equity and 
most likely would not have been able to 
answer a challenge under the Canadian 
Human Rights Act. 


5.88 Though job evaluation with the 
Standard will be simplified because there 
will be a single classification standard 
instead of the 70 now used, the cost to 
operate the job evaluation system will 
remain high, particularly if detailed job 
descriptions continue to be used to collect 
information about jobs. We believe that 
more effective and less costly options 
exist that could significantly reduce the 
operating costs of job evaluation. In times 
of constraint, such opportunities must be 
sought vigorously. Further reduction of the 
operating cost of job evaluation would 
ensure greater support from departments. 


5.89 Unless there are changes to the 
proposed occupational group structure, 
simplification is likely to be significantly 
less than recommended by the task force 
or anticipated and desired by many 
departments for optimum efficiency. 
Departments agreed, however, that even 
the proposed structure would be an 
improvement over the current 
occupational group structure. Significant 
Opportunities exist to further simplify the 
structure to take into account the need for 
improved departmental efficiency and/or 
reduce the cost of some of the associated 
systems and practices, such as pay 
administration. If not enough flexibility is 
provided to departments, the cost and 
benefits of the reform — in dollars as well 
as in human terms — may be spread 
unevenly. 


5.90 While the Treasury Board 
Secretariat has undertaken the reform with 
a commitment to “cost-neutrality”’, this 
objective was never clearly defined and 
was possibly unrealistic. At the conclusion 
of our audit it was not yet in a position to 
assess with any degree of precision the 
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cost of the current systems or the cost and 
benefits of the reform. 


The Standard Has Potential and 
Can Be Made to Work 


5.91 In our opinion, the Standard is a 
step in the right direction and its design 
represents an improvement over the many 
job evaluation plans now used by 
departments. The promising features of 
the Standard, the very significant 
developmental work accomplished to date 
and the general satisfaction of most 
departments lead us to conclude that the 
Secretariat should pursue its efforts to 
develop the Standard to meet both the 
objectives of the reform and the 
requirements of the Canadian Human 
Rights Act. All job evaluation or pay 
equity experts we consulted agreed that, 
although tough technical issues need to be 
addressed to ensure that the Standard is 
valid and credible, these difficulties can 
be overcome with commitment by all 
stakeholders, time, resources and 
improved project management. 


Important Measures Are Being 
Taken to Address Issues 
Identified 


5.92 At the conclusion of our audit, 
the Secretariat was taking a number of 
important measures to address issues that 
we, and other stakeholders, have 
identified. Although initially they reacted 
slowly to our concerns about the 
Universal Classification Standard, we 
have noted since the fall of 1995 a 
significant change in attitude among 
Treasury Board Secretariat’s officials in 
relation to our findings. This change has 
paved the way for improvement. As 
mentioned, the Secretariat has taken 
several measures to obtain advice and 
points of view from various stakeholders 
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and experts. In addition, important work 
was being carried out diligently with an 
objective of ensuring that the Standard 
will be a valid and reliable instrument 
capable of evaluating all jobs in the public 
service, while meeting the requirements of 
the Canadian Human Rights Act. 
Although it is too early to assess the net 
effect of the many measures being taken, 
they are encouraging. 


In our opinion, the 
Standard is a step in 
the right direction and 
its design represents 
an improvement over 
the many job 
evaluation plans now 
used by departments. 


The Challenge Ahead 


The Key to Success: 
Stakeholders Working Together 


5.93 To a large extent, the success of 
the reform depends on the willingness and 
ability of the various stakeholders to work 
together on developing an effective tool 
for job evaluation and reducing the cost of 
the system. However, because 
classification and job evaluation impact 
significantly on other human resource 
systems and on the efficiency of program 
delivery in departments, it is the whole 
approach to human resource management 
in the public service that is being 
challenged by this reform. While the 
initial focus may be job evaluation and the 
simplification of the occupational 
structure, the reform cannot be a 
piecemeal effort. A significant number of 
related issues will need to be addressed, 
such as: 


Difficulties can be 
overcome with 
commitment by all 
stakeholders, and with 
time, resources and 
improved project 
management. 


e accommodating the need for 
departmental flexibility while ensuring 
that job evaluation is managed 
consistently across departments in which 
benefits and costs — in human terms, as 
well as in dollars — may be spread very 
unevenly; 


e negotiating salaries in the future and 
simplifying pay administration in a way 
that maintains internal relativity in the 
public service while ensuring that people 
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Simplification is 
necessary to improve 
efficiency and 
flexibility and reduce 
administrative costs, 
in order to free up the 
financial resources 
needed to implement 
the reform while 
continuing to provide 
quality of service to 
Canadians at a 
reasonable cost. 


whose skills are at a premium in the 
labour market are attracted and retained; 


e staffing under a simplified 
occupational group structure while 
considering the need for specialized 
knowledge, skills or experience; and 


e reconciling departments’ 
accountability for results and efficient 
program delivery, given their limited 
degree of latitude in such matters as job 
evaluation, the occupational group 
structure, collective bargaining and pay. 


Reform at a Reasonable Cost to 
the Taxpayer 


5.94 Simplification is necessary to 
improve efficiency and flexibility and 
reduce administrative costs, in order to 
free up the financial resources needed to 
implement the reform while continuing to 
provide quality of service to Canadians at 
a reasonable cost. One reason for the 
reported success of the simplification of 
classification and job evaluation in 
Australia was, according to Australian 
authorities, the strong incentive provided 
by the direct link between compensation 
and improvement in overall productivity. 
In Australia, labour legislation governing 
the public service and the “structural 


efficiency principle” require that increases 
in pay rates be linked to overall 
productivity improvement. In the current 
context of restraint, there might be merit 
in such an approach. 


Treasury Board Secretariat’s comments: 
Treasury Board Secretariat agrees with the 
Auditor General that classification and 
Job evaluation are considered “the 
cornerstone of human resource 
management in the public service”, that 
“the status quo is unacceptable” and that 
the “standard has potential and can be 
made to work”. 


Human resource management systems use 
departmental resources (including 
management time) — some of which might 
be better targeted at providing quality 
services to taxpayers and reducing the 
deficit. However, to achieve significant 
efficiencies in all these human resource 
systems, we need first to modernize how 
we value work and classify jobs. 


TBS is committed to working 
co-operatively with all of the principal 
stakeholders to try to achieve one simple, 
consistent, affordable work evaluation tool 
that is capable of evaluating all public 
service work in a consistent, bias-free 

way, and that reflects new public service 
values. 
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y About the Audit 


Objective 


We wanted to assess whether the development and design of the Universal Classification Standard was 
appropriate to achieve critical objectives of the reform, such as universality (applicable to the public service 
as a whole) gender neutrality (free of bias), and simplification. We also wanted to assess whether the proposed 
occupational group structure would support efficient program delivery, and how thoroughly Treasury Board 
Secretariat had taken into account the cost/benefit factor. 


Scope 


Our audit included work performed by the Secretariat to: design and develop tools to collect information on 
job content; design and develop the new job evaluation plan; determine the relative importance of each 
element used to evaluate jobs (the “weighting’’); test and validate the new plan; apply the new job evaluation 
plan to a number of positions; and simplify the occupational group structure. 


Our audit of the simplification and operational costs of job evaluation as well as of the occupational group 
structure consisted of a review and an evaluation of the representations made by departments and unions, and 
of the work and studies conducted in the context of the task force or by Treasury Board Secretariat. 


We interviewed deputy or assistant deputy ministers or directors general of human resources in 
_ 10 departments, ranging from large to small. These departments represented more than 70 percent of the 
employees proposed for inclusion in the new General Services group. 


We visited six private and public sector organizations that had recently developed a new job evaluation 
system and that, in many cases, had simplified their occupational group structure, to assess how they had gone 
about it. We also conducted numerous interviews and consultations with nine other organizations and three 
compensation management consulting firms. 


We also examined documentation on classification reform in Australia’s public service, and recent literature 
in the field of job evaluation and work design. 


Criteria 


Our examination of the development and design of the Universal Classification Standard used recognized 
criteria in the field of job evaluation and took into account the government’s objectives for the new system. 


In the absence of specific criteria for gender neutrality, we developed our own from a variety of sources, 
including: the Canadian Human Rights Act, the Equal Wage Guidelines, 1986, a publication by the Canadian 
Human Rights Commission entitled Implementing Pay Equity in the Federal Jurisdiction, rulings by the 
Ontario Pay Equity Review Tribunal and the Canadian Human Rights Tribunal, and academic research on job 
evaluation and gender neutrality. Treasury Board Secretariat officials and pay equity experts reviewed our 
criteria, and changes were made accordingly. Officials from the Canadian Human Rights Commission found 
our criteria accurate and appropriate. A number of pay equity experts provided us with opinions on the 
universality and gender neutrality of the Standard. 
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Appendix 


SUMMARY OF SIGNIFICANT DEFICIENCIES IN THE VARIOUS COMPONENTS 
OF THE UNIVERSAL CLASSIFICATION STANDARD 


Elements 


e The elements—definition component is intended to explain what the element is supposed to measure. Except for few 
elements, definitions are vague, broad and at times ambiguous. For example: 


- “Thinking Challenge” is defined as measuring “the effort of the thinking challenge in terms of complexity of the 
demands-and outcomes associated with the work performed.” This definition does not explain what a “demand” 
and an “outcome” are. Furthermore, it is not clear if the demands and outcomes always vary together in 
complexity. Although Notes to Evaluators are intended to provide guidelines on how each element is to be 
applied, they do not provide enough instructions to make sure the evaluator has the appropriate interpretation of 
what is measured. 


— “Care of Individuals” is intended to measure “the responsibility to provide direct care to individuals....” It is not 
known what types of direct care are measured, such as short-term, long-term, critical or non-critical and how these 
factors should or should not be taken into consideration. Notes to Evaluators include no additional explanation to 
assist the evaluator in interpreting the purpose of the element. 


Degrees and Their Definitions 


e Degree definitions have been defined on a rating scale to represent a progression from a minimum to a maximum value. 
In their present form, degree definitions are extremely vague and ambiguous. In fact, except for two elements, degrees 
are not really defined. They have been framed in “generic” words such as some, moderate, significant, extensive, etc. 
and they cannot assist the evaluator in establishing meanings for these words or providing rules of differentiation 
among jobs. For example: 


— For the “Physical Demands” element, “Limited effort” was defined as “Limited physical effort is required.” 
Degree Illustrations _ 


e Degree illustrations serve to exemplify degrees in concrete job content and to show the progression from one level to 
another. Because element definitions and degree definitions are too general and broad, degree illustrations constitute the 
actual criteria against which the job is evaluated. Evaluators rate jobs mainly by examining various degree illustrations 
at different levels and then deciding the “best fit” for the job evaluated. The actual scales embedded in the degree 
illustrations drive ratings. As it stands, degree illustrations are the critical component in the Standard and they contain 
the most serious shortcomings. 


e The range and breadth of elements are undermined by degree illustrations. In many cases, the criteria included in them 
are different from those in the element definition and degree definition. For example: 


— The “Thinking Challenge” element is supposed to measure the complexity of demands and outcomes, while the 
primary difference between degree illustrations at different levels is in scope of authority. 


e The criteria embedded in degree illustrations in several elements concentrate too much on the same 
managerial/hierarchical aspect of work, resulting in too narrow a range of job content. There is too much emphasis on 
organizational placement, which is the major discernible difference among degrees in elements, even ones that should 
not be measuring scope of authority. 


e Some aspects of work are missing, particularly aspects of women’s work and work at a high level such as complex | 
collegial decision making, the responsibility of maintaining confidential and sensitive information, the requirement to | 
keep up-to-date in technology, the effort and skills to negotiate without authority, human relations skills, etc. 


e Based on degree illustrations for some elements, what is measured and the progression from one degree to another are | 
unclear. For example: | 


— The element “Influence” aims at measuring “the extent to which activities, decisions or recommendations 
influence the service being delivered.” Based on degree illustrations, it is not clear what type of influence is being 
measured, i.e. direct, indirect, contributory, shared or other. 
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e Consistency of degree illustrations is problematic in several cases: within the same degree, there are non-equivalent degree 
illustrations, and between two degrees there are illustrations with no discernible differences. For example: 


— The element “Context” aims at measuring “the extent to which knowledge of the context is required to perform the 
work.” The following degree illustration, “The work involves scheduling the output of a data processing centre and 
determining application specifications, service levels and equipment resource requirements. It requires knowledge of the 
organization and work flow of the centre and the various operational requirements of the centre’s client”, is clearly not 
equivalent to another: “ The work involves conserving and protecting the fishery resources in a sub-district of a region 
and requires knowledge of the mandate and objectives of the department. It also requires knowledge of the fishing 
industry, the geography of the area, and the jurisdictional relationship within the commercial, sport and native 
fisheries.” Yet both are at the same degree level. : 


e Degree illustrations systematically devalue women’s work, notably by placing job content of female-dominated jobs at 
inappropriately low levels compared with male-dominated jobs. For example: 


— In the “Thinking Challenge” element, the degree illustration, “ The work involves changing dressings to wounds or 

incisions, taking vital signs, observing and recording changes in condition of patients, reporting any equipment failures 
and assisting in arranging clinics and scheduling opportunities” was placed at the same degree level as ““ The work 
involves establishing cleaning priorities, deciding on the types or strengths of chemical cleaning products to be used 
and monitoring the quality of services provided by other cleaners.” 


e Another way degree illustrations systematically devalue women’s work is by having degree illustrations that do not 
adequately differentiate job content at various degree levels among female jobs. For example: 


— In the “Care of Individuals” element, the two nursing jobs in the following degree illustrations have been placed at the 
same degree levels but are very different in terms of responsibility. The degree illustration, “The work involves 

providing nursing care to patients in a hospital. Direct care includes carrying out therapeutic measures and diagnostic 

| tests, monitoring symptoms and reactions to treatment and monitoring the progress of patients” requires greater 

| responsibility than the nurses’ aide, “The work involves caring for patients. Direct care includes feeding and providing 

basic hygiene, remedial treatments, comfort measures such as skin care and emotional support.” This demonstrates that 

this element does not differentiate very well among the various degree levels for female jobs. 


e Instructions given in Notes to Evaluators to credit specific job content are not always reflected in degree illustrations, which 
means that the job content in those cases is unlikely to be credited by evaluators. For example: 


~ For the element “Influence”, the notes suggest considering the responsibility for independent action, or as a team 
member. This was intended to recognize frequently overlooked female job content, collegial decision making. There is, 
however, no degree illustration that credits such a responsibility. 


° A qualitative examination of degree illustrations revealed significant overlapping. Statistical analysis was consistent with this 
observation and showed a high correlation for 8 out of 16 elements. This has the effect of limiting the universality of the 
Standard. The more overlapping there is, the higher is the risk that not enough attention will be paid to the full range of job 
content in an organization. 


¢ — Statistical tests have been performed to diagnose for systemic effects by gender. One of the tests revealed that 6 of the 16 
elements of the Standard measure male and female job content equally well. The 10 remaining elements favour the content 
of male-dominated jobs. These results are indicators of potential bias in the Standard. 


Examples are drawn from the June 1995 version of the Universal Classification Standard issued by the Treasury Board Secretariat to departments. 
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Preface 


The Policy Backdrop 


The 1990s have so far proven to be a significant watershed in world politics, leading to a fundamental 
restructuring of international affairs. Beginning with the demise of the Soviet Union and the disbanding of the 
Warsaw Pact, the first half of the decade has seen Canada reorient its security concerns from the Cold War to 
attempting to restore order to states such as Somalia, the former Yugoslavia and Haiti. As the focus of security 
shifted, the Canadian government undertook a major policy review to manage change. 


During the spring and summer of 1994, Parliament established two special joint committees — one on 
foreign policy and the other on national defence — to examine Canada’s role in the world and the place of 
peacekeeping in Canada’s priorities. The establishment of these parliamentary committees was the first step to 
drafting new foreign and defence policy white papers. Taken together, the work of the special joint committees 
and the two white papers provide the backdrop for current peacekeeping operations. 


The Special Joint Committee Reviewing Canadian Foreign Policy took a broad view of security and put 
considerable focus on the importance of co-operation on development issues in the prevention of international 
crises. The Committee advocated bringing economic and military measures together into an integrated approach 
to security problems. It also favoured a United Nations rapid reaction force to provide assistance in a more timely 
manner, and a Canadian peacekeeping training centre. According to the Committee, the “primary international 
role” of the Canadian Forces should be peacekeeping operations. 


The Special Joint Committee on Canada’s Defence Policy commented on the need to improve the 
capability of international institutions such as the United Nations. The Committee asked for “clear criteria” 
regarding Canadian participation in peacekeeping missions. After hearing testimony on the problems faced by the 
land forces in providing relatively large numbers of troops to the United Nations on a continuing basis, the 
Committee recommended increasing the land forces by 3,500 combat troops. 


The Foreign Affairs and Defence white papers published by the government later the same year adopted 
similar approaches to peacekeeping. Both white papers noted the problems of the United Nations in acting 
effectively and included calls for a United Nations rapid reaction capability, improvements in decision-making 
processes and sounder financing. Both papers promised continued military assistance to the United Nations, 
including headquarters staff, and both noted with approval the establishment of the Pearson Peacekeeping Centre. 


Each of the two white papers also had unique elements. The Foreign Affairs white paper repeated the 
theme of security that goes beyond military preparedness and mentioned sustainable development, preventive 
diplomacy and co-operation on development issues as elements in Canada’s approach to dealing with instability. 
The Defence white paper focussed on the government’s military policy and stated the government’s commitment 
to a range of military options including preventive deployments, peacekeeping and observer missions, “enforcing 
the will of the international community”, and post-conflict peace-building and confidence-building measures. 


These intertwined and mutually supporting policies form the basis for current Canadian peacekeeping 
activities. The two departments that are primarily charged with carrying out government policy are also deeply 
meshed together. 
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Canadian Government Organization 


Government decision making. As explained in the Foreign Affairs chapter, decisions about whether or 
not Canada should join a new peacekeeping mission are usually made at the Cabinet level. It is customary that the 
document on which Cabinet bases its decision is signed by both the Minister for Foreign Affairs and the Minister 
of National Defence. The Minister of National Defence and the Chief of Defence Staff are the sources of advice 
on military considerations for the Cabinet. What this means in practice is that officials in both departments must 
agree before the request from the United Nations for Canadian troops may be considered by Cabinet. Foreign 
Affairs usually writes the first draft. 


Interdepartmental co-operation. It appears from our audit that Foreign Affairs and National Defence 
officials work closely together. The Foreign Affairs chapter gives a number of examples of the departments’ 
co-operation, such as the Canadian study on a rapid reaction capability for the United Nations. Although these 
efforts appear to have been almost always beneficial, we found that the co-operation has not produced optimum 
results in the area of financial administration. 


Personnel interchange. The interchange of personnel between the two departments in key positions 
relating to peacekeeping has been beneficial in policy making and implementation. The official directly 
responsible for peacekeeping in Foreign Affairs is a Canadian Forces officer on secondment from National 
Defence. Three Canadian Forces personnel work at the Canadian Permanent Mission to the United Nations in 
New York. 


When a conflict is developing and the governments of the world are considering whether and how the 
United Nations should be involved in efforts toward peace, the Canadian Mission to the United Nations plays a 
key role in presenting the views of the Canadian government and keeping the government closely informed about 
the ensuing discussions. Communications back to both departments appear to work well during this critical phase. 


The Following Audit Chapters 


These two audit chapters attempt to assess Canada’s management of its policies as stated in the two white 
papers. The Foreign Affairs chapter focusses on the adequacy of information to Parliament, information for 
decision making, efforts toward reform at the UN, the cost of peacekeeping, and financial management. We 
reported on National Defence costs and related aspects of administration in the Foreign Affairs chapter because an 
important role of National Defence in peacekeeping is to support the delivery of Canada’s foreign policy, and 
relations with the UN are conducted through Foreign Affairs and International Trade. The National Defence 
chapter examines the ability of the Department to train and field peacekeeping forces within the limits set by the 
Defence white paper. 


We expect that a number of benefits will result from these audits, including the following: 


e There will be enhanced transparency, comprehensiveness and quality of information provided to 
Parliament and to the public about Canada’s participation in peacekeeping. 


e The roles, responsibilities and accountabilities of the key players involved will become clearer. 
e The relevant management processes will be streamlined. 


e The implications beyond the financial cost of peacekeeping as an activity of the government will be 
even more openly debated and the importance of good financial management better recognized. 


e There will be a greater appreciation of the overall cost to Canada of peacekeeping, especially in view 
of the present fiscal realities. 
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Peacekeeping — Foreign Affairs 
and International Trade Canada 


The audit work reported in this chapter was conducted in accordance with the 
legislative mandate, policies and practices of the Office of the Auditor General. 
These policies and practices embrace the standards recommended by the Public 
Sector Accounting and Auditing Board (PSAAB) of the Canadian Institute of 
Chartered Accountants. 
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Peacekeeping — Foreign Affairs 


and International Trade Canada 


Assistant Auditor General: Richard B. Fadden 
Responsible Auditor: Vinod Sahgal 


Main Points 


6.1 Foreign Affairs and International Trade Canada views peacekeeping as central to Canada’s foreign policy 
objective of protecting Canada’s security within a stable global framework. Over the years, Canada has 
participated in most peacekeeping missions. 


6.2 The nature of UN peacekeeping has changed dramatically in the post-Cold War era. Before, the main role 
of the United Nations (UN) was largely to monitor truce agreements between states. Now, the UN is increasingly 
called upon to carry out more complex missions that also involve a peace enforcement role both between and 
within states. As well, a role for civil-military co-ordination and co-operation has emerged. 


6.3 The United Nations continues to be the key vehicle for pursuing Canada’s global security objectives; 
however, it faces significant constraints in its ability to carry out its peacekeeping function effectively: 


e The UN’s management points out that member states collectively have not been prepared to 
adequately equip the organization to meet the additional challenges, as shown by experience in 
Somalia, Rwanda, Bosnia and Haiti. 


e The Department points out that missions with unclear mandates and inadequate resources are 
unlikely to succeed, despite improvements in management practices at UN headquarters. 


e Unlike Canada, a number of members are not paying their dues on time, with the result that 
peacekeeping funds are increasingly being used to defray the cost of the regular budget, which is also 
seriously in arrears. 


6.4 Canada places a high priority on reform of the UN, and has worked on several proposals to improve the 
UN’s effectiveness in peacekeeping. However, significant reforms require consensus of the 185-member General 
Assembly. Consensus on change has proved difficult to find. 


6.5 The overall cost of peacekeeping to Canada, both financial and non-financial, is not aggregated in any 
public document. The financial costs have risen sharply in recent years, as a result of the increase in the number 
and size of missions, from about $47 million in 1991-92 to an estimated $240 million in 1995-96, an increase of 
410 percent. The cost of participation in NATO peacekeeping in Bosnia and Canada’s recent decision to send 
troops to Haiti will add to this increase. 


6.6 The Department has no established procedure to carry out evaluations or “lessons learned” exercises in 
the area of peacekeeping from a foreign policy perspective. This increases the risk of not benefiting fully from the 
experience gained from previous participation. 


6.7 Canada has recovered $236 million of its peacekeeping costs from the UN during the last five years. 
Reimbursements estimated at $92 million were receivable at 31 March 1996. The amount and timing of 
reimbursement recovery is largely dependent on the UN’s cash position, the priority Canada gives to the collection 
function, invoices and supporting documentation and acceptance by the United Nations of amounts billed. 


6.8 Informed decision making on peacekeeping matters would be enhanced by greater transparency and 
accountability. Also, the administration for reimbursements from the UN needs to be strengthened. 
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Introduction 


Canada’s Foreign Policy and 
Peacekeeping 


6.9 The Cold War period 
(1946-1990) was dangerous but it is 
generally believed that events during that 
period were relatively predictable. 
Canada’s memberships in the North 
Atlantic Treaty Organization (NATO) and 
the North American Aerospace Defence 
(NORAD) Agreements were key 
guarantees of Canada’s military security. 
In the post-Cold War period, the 
perception of the old external threat posed 
by superpower confrontation has for all 
practical purposes diminished. Although 
direct threats to Canada’s security have 
diminished, the international security 
environment is vastly more complex and a 
whole range of issues such as the 
environment, overpopulation and 
increasing disparity have peace and 
security implications at the local, regional 
and global levels. 


6.10 Foreign Affairs and International 
Trade Canada is responsible for 
developing Canada’s foreign policy, 
conducting Canada’s relations with the 
United Nations and, with the Department 
of National Defence, it is responsible for 
advising the government whether and to 
what extent Canada should participate in 
specific UN peacekeeping missions. 
Canada, like any other member state, is 
obliged to pay its assessed share of UN 
peacekeeping costs whether or not it 
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actually takes part in a mission. However, 
participation is an option and the 
Department’s advisory role is important in 
this area. The Department works closely 
with National Defence and other 
government departments in this role. 
National Defence is also responsible for 
planning Canada’s participation at the 
operational level, and providing the troops 
and equipment. The responsibility for 
financial management is shared by the two 
departments. 


6.11 For a highly trade-dependent 
country like Canada, prosperity and 
employment at home require Canada to 
co-operate with other nations to attract 
investments and to promote Canadian 
goods and services in overseas markets. 
Stability and security are prerequisites for 
the economic growth and development of 
these markets. Thus the objectives of 
promoting prosperity and employment and 
promoting global peace are interdependent 
and mutually reinforcing. 


6.12 Canada’s commitment to 
international peace and security was 
reaffirmed in the government’s 1995 
foreign policy statement “‘ Canada in the 
World”. (See Exhibit 6.1.) The promotion 
of global peace as the key to protecting 
Canada’s security remains a central 
element of our foreign policy. Canadian 
security, including economic security, is 
viewed as increasingly dependent on the 
security of others. Protecting Canada’s 
security goes beyond military 
preparedness. A broad concept of security 
is needed to address serious challenges, 


three key objectives: 


The government's foreign policy statement of March 1995, “Canada in the World”, identifies 


@ Promotion of prosperity and employment 
® Protection of Canada’s security within a stable global framework 


@ Projection of Canadian values and culture 


The promotion of 
global peace as the 
key to protecting 
Canada’s security 
remains a central 
element of our foreign 
policy. 


The UN continues to 
be the key vehicle for 
pursuing Canada’s 
global security 
objectives. 


Exhibit 6.1 


Canadian Foreign Policy 
Objectives 


Source: “‘Canada in the World” 
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Canada has been 
involved in most 
peacekeeping 
operations, both UN 
and non-UN, since the 
end of the Second 
World War. 


Canada’s monument to 
peacekeeping located in 
downtown Ottawa (see 
paragraph 6.14). 


Source: Department of 
National Defence 


including environmental and human 
tragedies from which Canada, even half a 
world away, is not immune. This broader 
orientation requires approaches that go 
beyond military options and focus on the 
economic, social and political causes of 
conflicts. 


6.13 The government has reaffirmed 
that the “UN continues to be the key 
vehicle for pursuing Canada’s global 
security objectives” and “the success of 
the UN is fundamental, therefore, to 
Canada’s future security.” Canada 
recognizes that the UN must be more 
effective in order to be fully capable of 
dealing with the array of new global 
security issues. With this in mind, Canada 
is pressing for improvements to strengthen 
the UN’s peacekeeping and other 
functions. 


Canada’s Contributions to 
Peacekeeping 


6.14 Canada has been involved in 
most peacekeeping operations, both UN 
and non-UN, since the end of the Second 
World War. Examples of important 
non-UN operations are the two Indo-China 
peace-monitoring missions in the 1950s to 
the 1970s in which Canada participated. 
Canada has contributed over 100,000 
troops to 30 of 36 UN peacekeeping 
missions to date, and its role as a UN 
peacekeeper is respected internationally. 


6.15 Over the years, Canada has 
provided military personnel and 
equipment, as well as civilian 
peacekeepers (election monitors, civilian 
police, human rights monitors and 
sanction monitors). As of 

31 January 1996, Canada was 


6-8 


Report of the Auditor General of Canada — May 1996 


participating in 10 of 19 UN peacekeeping 
operations, contributing over 850 out of a 
total of about 29,000 UN troops. In 
addition, about 100 Canadian police 
officers were engaged in civilian police 
duties in Haiti. Of the 70 member states 
who are currently contributing troops and 
civilians, Canada is currently the eleventh 
largest contributor. Canada has also 
contributed 1,000 troops to the NATO-led 
Implementation Force (IFOR) in the 
former Yugoslavia and is sending up to 
750 troops to the UN mission in Haiti, at 
its own cost, to replace returning 
American troops. 


6.16 Nine Canadian military staff are 
on loan to the UN, at Canada’s cost, for 
peacekeeping-related duties. Canada has 
organized specialized meetings and 
conferences, and funded peacekeeping 
training. It also continues to initiate 
improvements to the UN’s effectiveness in 
peacekeeping, such as the study on a rapid 
reaction capability and another study on a 
revised budgetary and reimbursement 
system for peacekeeping operations. 


The Nature of Peacekeeping Has 
Changed 


6.17 The nature of peacekeeping has 
changed dramatically in the post-Cold 
War period. Between 1945 and 1987, the 
UN’s main role in peacekeeping was 
generally confined to low-level 
monitoring of peace agreements reached 
between conflicting states. The situations 
encountered were generally quite stable 
and did not involve the deployment of 
large numbers of troops or equipment. 


6.18 Since the end of the Cold War, 
there has been a proliferation of so-called 
complex emergencies entailing a 
breakdown of legitimate institutions and 
governance, widespread suffering and 
massive displacements. The potential for 


—_————— 
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“new violence” of a more complex and 
changing nature has emerged. 


6.19 These new situations often 
involve and require a range of responses 
from the international community within a 
single mission. Not only is a military 
presence necessary to monitor a peace 
agreement, if one exists, but major efforts 
increasingly are needed to provide 
humanitarian, medical and disaster relief; 
contain and rehabilitate refugees; monitor 
human rights abuses; enforce sanctions; 
and supervise elections. Peacekeeping in 
the current context is much broader and 
includes peacebuilding, peacemaking and 
peace enforcement. Often peace must first 
be established before it can be enforced 
and kept. Thus the complexity of 
mandates has increased significantly. 


6.20 Civil organizations, such as 
non-governmental organizations (NGOs), 
have come to play an increasingly larger 
and more important role in relief and 
humanitarian activities in conflict areas, 
either on their own initiative (through 
funding provided by bilateral donors) or 
on behalf of the UN. This, of course, 
applies as much or more to the 
international development agencies (such 
as CIDA) that fund many of these NGOs. 


6.21 Given the important role that 
NGOs are increasingly being asked to 
play, a need has emerged for the effective 
co-operation and co-ordination of civilian 
and military efforts to deal concurrently 
with all important aspects of security, 
including humanitarian concerns that 
underlie the conflicts. 


6.22 In the changed circumstances of 
“new” peacekeeping, the UN has had to 
assume far broader roles and objectives 
and conduct more complex operations 
than it is equipped for and, we were told, 
than its founders may have had in mind 
when it was conceived. Three recent 
examples of these challenges are the mass 


The nature of 
peacekeeping has 
changed dramatically 
in the post-Cold War 
period. 


In the changed 
circumstances of 
“new” peacekeeping, 
the UN has had to 
assume far broader 
roles and objectives 
and conduct more 
complex operations 
than it is equipped for. 
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United Nations 
headquarters in New York. 


Source: UN photo 


migration and starvation resulting from 
civil war in Somalia, the violent eruption 
of long-suppressed ethnic and religious 
divisions in the former Yugoslavia and the 
genocidal conflict based on tribal rivalries 
in Rwanda. 


6.23 The demand on the UN has 
grown tremendously, with 18 new 
peacekeeping missions established since 
1990, compared with 18 in all the 
previous years. UN troops in the field 
increased from 7,000 in 1989 to a peak of 
over 69,000 in 1994 and 1995, witha 
commensurate increase in annual 
peacekeeping expenditures from about 


$200 million to about $3.5 billion (US). 
Since then, however, as a result of 
handing over peace enforcement in the 
former Yugoslavia to the NATO-led force, 
the number of troops under the UN flag 
has dropped to about 29,000. 


The UN Faces Constraints in Its 
Peacekeeping Function 


6.24 The Security Council is the 
paramount UN decision-making body for 
peacekeeping. It consists of 15 members: 
5 permanent members who have a veto 
(China, France, Russian Federation, 
United Kingdom and USA) and 
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10 non-permanent members who are 
elected for a two-year period on a 
rotational basis and do not have a veto. 
Canada has served as a non-permanent 
member, usually once every 10 years, and 
its ability to influence decisions must be 
viewed in this context. 


6.25 The UN’s management has 
pointed out that it faces significant 
constraints on its ability to cope with the 
demands of the “new” peacekeeping. 
Recent experience in Somalia, Bosnia and 
Rwanda has highlighted the need for 
improvements, including speedier 
decisions by the Security Council; clear 
and realistic mandates; adequate financial 
and other resources to discharge 
mandates; a rapid reaction capability; 
timely and effective troop deployment; 
improved planning ability, logistical 
support, deployment and management of 
large missions; and a better co-ordinated 
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and expanded flow of information and 
efforts. 


6.26 The Department points out that 
missions with unclear mandates and 
inadequate resources are unlikely to 
succeed despite improvements in 
management practices at UN 
headquarters. Canada places a high 
priority on reform of the UN, and has put 
forward several proposals to improve the 
UN’s effectiveness in peacekeeping. 
However, significant reforms require the 
unanimous support of the 185-member 
General Assembly. Consensus on the 
direction and extent of reform is very 
often difficult to reach. Without 
consensus, even sustained effort by any 
one state will yield limited results. 


6.27 The UN’s serious financial 
problems and the difficulties in its 
peacekeeping ability can be resolved, but 
only through building a consensus of 


The UN’s management 
has pointed out that it 
faces significant 
constraints on its 
ability to cope with the 
demands of the “new” 
peacekeeping. 


The Security Council in 
session (see paragraph 6.24) 


Source: UN photo 
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The UN’s management 
has stated that 
members generally 
have not been 
prepared to 
collectively equip the 
organization 
adequately to meet the 
additional challenges. 


Exhibit 6.2 
Amounts Owed to the UN as at 


31 December 1995 
(US$ millions) 


Source: United Nations 


political will, requiring nations to take a 
larger view of their ideological and 
strategic interests. In the UN’s operating 
environment, individual members, 
particularly those (including Canada) who 
are not members of the Security Council, 
have little influence to generate sufficient 
political will among the most influential 
members to implement and fund needed 
reforms. Canada has actively supported, 
and in some cases proposed, reform 
measures but, as mentioned, the prospect 
of their finding acceptance is limited. The 
UN’s management has stated that 
members generally have not been 
prepared to collectively equip the 
organization adequately to meet the 
additional challenges. 


6.28 Since 1990, much-needed 
improvements have been made in certain 
aspects of managing UN peacekeeping 
operations at its headquarters. Canada can 
take credit for some of them, such as the 
creation of a 24-hour Situation Centre in 
the Department of Peacekeeping 
Operations, largely in response to the 
difficulty faced by the UN field 
commanders in obtaining immediate 
decisions from UN headquarters. A 
Canadian currently heads the Centre. But 
these improvements in themselves are not 
sufficient to overcome the more 
fundamental constraints outlined above. 


6.29 Furthermore, the UN is 
reportedly facing a serious cash shortage 
as a result of non-payment by many 
member countries of both the regular 
budget and peacekeeping budget 
assessments. As of 31 December 1995, 
members owed US $2.3 billion — 

$564 million to the regular budget and 
$1.7 billion for the peacekeeping budget. 
Two of the five permanent members of the 
Security Council are among the biggest 
defaulters (Exhibit 6.2). There could be 
many reasons, both political and financial, 
why members fall behind or choose not to 
pay, including cash or foreign exchange 
shortage, dissatisfaction with the formula 
used to calculate assessed shares (scale of 
assessment), the linkage made by some 
members between their contributions and 
UN reforms they desire, and other 
considerations. 


6.30 Recently, the Secretary General 
has been forced to “borrow” unparalleled 
amounts of peacekeeping funds to pay 
regular budget expenses, such as staff 
salaries and other administrative costs. In 
addition, the UN’s management is 
reportedly considering cuts in 
headquarters staff to cope with a budget 
freeze. We understand that staff in 
Peacekeeping Operations would also be 
affected, which could negate the 
improvements made to date. Peacekeeping 


-UNRegular Peacekeeping 

Budget Budget Total 

* United States 414 821 1235 
* Russian Federation - 404 404 
Ukraine 4] 186 PE: 
Belarus 10 48 58 
Germany 0 24 24 
All others 99 235 354 
Total 564 1,738 2,302 


* permanent members of the Security Council 
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funds are provided by members 
specifically to cover the common costs 
associated with missions as well as to 
reimburse countries contributing troops. 
When peacekeeping funds are not 
contributed by members or are used by the 
UN for other purposes, troop contributors 
do not get paid their reimbursements due 
from the UN for troop costs and use of 
equipment. In the words of the UN 
Secretary General on 6 February 1996, “... 
the financial crisis has brought the UN to 
the edge of insolvency... the UN owes 
about US $1 billion to countries that have 
contributed troops and equipment for 
peacekeeping.” 


Observations 


6.31 To fulfill our audit objective of 
providing better information to Parliament 
on Canada’s participation in 
peacekeeping, we examined the role of 
Foreign Affairs and International Trade in 
the government’s decision-making process 
and the Department’s management 
practices as they relate to peacekeeping 
activities. We also examined the role of 
National Defence in the administration of 
UN financial transactions related to 
Canada’s peacekeeping involvement. Our 
statements about UN peacekeeping 
operations are based upon information 
provided to us, which we did not audit. 
The sources of that information are found 
in Appendix 1. Details on our audit 
objective, scope, criteria and approach are 
included at the end of the chapter in the 
section ““About the Audit”. 


Keeping Parliament Informed 


Better information on the objectives 
and nature of peacekeeping is needed 


6.32 Our two peacekeeping audits 
examined the information provided to 
Parliament at both the program and 
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individual mission levels. This chapter 
deals with the former. Foreign Affairs and 
International Trade Canada views 
peacekeeping as an important tool of 
foreign policy. 


6.33 Because peacekeeping is a 
long-standing, essential part of Canada’s 
foreign policy, and decisions relating to 
peacekeeping are significant, it is 
important that Parliament have reliable, 
relevant and comprehensive information 
about Canada’s participation. Besides 
having significant foreign policy 
implications, these decisions involve lives 
of Canadians and Canada’s image abroad, 
and are thus of the highest importance. 


6.34 A better-informed Parliament 
results in a higher standard of debate, 
which in turn contributes to decisions and 
policies by government that are more 
solidly based and better understood and 
supported by Canadians. We found that 
comprehensive information on Canada’s 
participation in peacekeeping is not 
available to parliamentarians in any one 
public document. We believe such 
information is important because of the 
importance of peacekeeping to Canadians. 


6.35 The External Affairs Act does not 
specifically deal with peacekeeping. The 
Estimates Part III represents the main 
reference document reporting on a 
department’s expenditures and is expected 
to provide a comprehensive picture of all 
aspects of a particular activity. 


6.36 We examined the Part IIIs of 
Foreign Affairs and International Trade 
Canada, National Defence and other 
government departments involved in 
peacekeeping. The Part IIIs of Foreign 
Affairs give a listing of Canada’s assessed 
shares, paid by the Department, for each 
UN peacekeeping mission where costs are 
still being incurred. However, they do not 
clearly articulate the objective of 
participating in specific missions. The 


The UN is reportedly 
facing a serious cash 
shortage as a result of 
non-payment by many 
member countries. 


Comprehensive 
information on 
Canada’s participation 
in peacekeeping is not 
available to 
parliamentarians in 
any one public 
document. 
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It is not easy to get a 
full and accurate 
picture of the financial 
cost of peacekeeping 
to Canada. 


The government needs 
to provide a yearly 
comprehensive report 
to Parliament on the 
objectives, results, 
significant options 
considered and 
implications of 
Canada’s participation 
in peacekeeping. 


Part IIIs of National Defence disclose its 
full and incremental costs for the missions 
to which Canada contributes troops. The 
Part IIIs of smaller contributing 
government departments usually make no 
specific mention of their role or 
contribution to peacekeeping. 


6.37 
information provided in any department’s 
Part IIs on either the amounts owed to 
Canada by the UN or the amounts 
recovered from the UN. In other words, it 
is not easy to get a full and accurate 
picture of the financial cost of 
peacekeeping to Canada. The information 
in Part IIs is designed to serve the 
departments’ own reporting objectives. 
However, even the Part IIIs of both 
Foreign Affairs and International Trade 
and National Defence considered together 
do not present a clear and comprehensive 
picture of all aspects of Canada’s 
participation in peacekeeping. 


We noted that there is insufficient 


6.38 In addition, the 1995-96 
Estimates Part III of National Defence 
contained some information that is not 
fully understandable. The disclosure of 
information in the Part IIIs needs to be 
revised to remove any ambiguity 
concerning the reporting of UN revenues. 
The assertion was made in the section 
“Peacekeeping and Related Operations” 
that amounts reimbursed by the UN are 
currently returned to the Consolidated 
Revenue Fund. We found that in addition 
to those reimbursements, there are other 
revenues, associated with Letters of Assist 
(contracts with the UN for the supply of 
goods and/or services), which are retained 
by the Department, thereby offsetting a 
portion of its incremental costs (see 
paragraph 6.106). National Defence also 
states in its Part IIIs that Canada is 
eligible for reimbursement by the UN for 
death and disability compensation. We 
noted that Canada has not billed the UN 


for death and disability payments (see 
paragraph 6.109). 


6.39 We also noted that neither the 
Part IIIs of Foreign Affairs and 
International Trade nor those of any other 
department mention the full implications 
for Canada of its participation in 
peacekeeping. Some implications brought 
to our attention by the Department include 
opportunities for Canadian business and 
the impact of UN duties on ongoing 
training and the well-being of Canadian 
troops (see Chapter 7 on National 
Defence). Information is also not provided 
on whether other ways of contributing to 
UN peacekeeping missions were 
considered, for example, other mixes of 
military and civilian personnel, fewer 
troops and more humanitarian aid. 
Although there is some description of 
Canadian efforts in trying to improve the 
UN’s peacekeeping function, neither these 
efforts nor the potential for results are 
described adequately. 


6.40 In view of the above limitations, 
we believe that the government needs to 
provide a yearly comprehensive report to 
Parliament on the objectives, results, 
significant options considered, costs and 
implications of Canada’s participation in 
peacekeeping. Our 1994 Report Chapter 7 
on “Information for Parliament” pointed 
out the importance of a lead department 
for reporting in situations where more 
than one department is involved in a 
particular activity. The section on Search 
and Rescue in the Part IIs of National 
Defence serves as an example of how this 
can be done. We believe that the lead 
department for providing information to 
Parliament could be Foreign Affairs and 
International Trade. A primary purpose of 
participating in peacekeeping is linked to 
Canada’s foreign policy. 


6.41 The Department is concerned that 
by assuming the role of lead department 
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for reporting purposes, it would be held 
accountable for all peacekeeping 
expenditures, such as incremental costs of 
National Defence, rather than only for its 
designated current responsibilities, that is, 
paying Canada’s assessed share. We were 
informed by departmental officials that 
the capacity for assuming a wider 
management responsibility does not 
currently exist at the Department. What 
we envision is a lead department 
designated to bring together information 
on all significant aspects of peacekeeping 
and report it to Parliament without 
necessarily changing respective 
accountabilities. Nonetheless, the 
Government of Canada needs to decide on 
how best to provide the above-noted 
information to Parliament. 


Information for Decision Making 


Better information on implications and 
results is needed 


6.42 Our objective in examining this 
aspect was to assess the nature and quality 
of information received by ministers for 
making decisions about Canadian 
participation in peacekeeping missions. 


6.43 Decisions on whether and to what 
extent Canada should participate in UN 
peacekeeping missions are normally made 
at the Cabinet level. 


6.44 We believe that information for 
decision making needs to include: 


e aclear statement of the nature and 
extent of participation and the potential 
for achieving Canadian foreign policy 
objectives; 


e analysis of the political, 
humanitarian and military situation in the 
country/region of conflict; 
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* an assessment of the physical risks to 
Canadian personnel and of the probable 
duration of involvement; 


e the financial cost and other 
implications for Canada: 


e an assessment of whether 
government guidelines for participation 
are being followed; and 


e the different ways in which Canada 
could participate, and an assessment of the 
lessons learned from participation in 
previous missions. 


6.45 In examining this issue, we 
studied all relevant documents we could 
obtain for the UN peacekeeping missions 
in Cambodia, Somalia, Rwanda, former 
Yugoslavia, and Haiti. We had access to 
all the documents that were available and 
obtained sufficient information to assess 
the quality of information to ministers on 
matters of foreign policy. We were 
informed by officials that ministers also 
receive information by oral briefings and 
from other sources, which we could not 
audit. 


6.46 Although documents specify the 
nature and extent of the role of Canadian 
troops in support of a UN mission, the 
specific benefits that Canada would derive 
over and above supporting the UN are not 
always stated. For example, other benefits 
that would accompany the cessation of a 
conflict, in addition to stability and 
humanitarian aspects, could range from 
political benefits vis-a-vis important 
trading partners to avoiding possible 
refugee flows to Canada. As a result, it 
was difficult to assess the basis on which 
the nature and extent of participation was 
recommended to ministers. In the absence 
of a clear statement on what Canada 
additionally hoped to achieve from a 
specific mission, it was presumed that 
Canada’s objectives were identical to 
those of the UN. The UN’s objectives, 
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expressed in terms of one or more 
Security Council resolutions (which 
inevitably reflect the compromises 
necessary to reach consensus), are often 
unclear. 


6.47 One example of an unrealistic 
mandate was the UN Security Council’s 
passing of a resolution that declared 
certain cities in Bosnia to be “safe areas”. 
In retrospect, it is clear that insufficient 
thought was given to the force necessary 
to implement such a mandate; the Security 
Council resolution was an expression of 
hope that, on the ground, could not be 
implemented. Foreign Affairs officials 
have also informed us that, in the case of 
the UN’s protection force in 

ex- Yugoslavia, UNPROFOR, several 
resolutions were passed providing 
additional, sometimes conflicting, tasks 
for the UN forces that made the overall 
mandate unclear. 


6.48 Ministers are normally provided 
with a reasonable analysis of the political 
and humanitarian situation in the 
country/region of conflict. Usually there is 
also a brief statement on the degree of risk 
that Canadian soldiers would face. 


6.49 An area of risk for which little or 
no information is usually documented for 
ministers is the potential for longer-term 
involvement of Canada beyond the usual 
six-month mandate given to UN missions 
by the Security Council. It is extremely 
rare, if ever, that UN missions terminate in 
six months. Most continue for much 
longer periods. For example, the 
monitoring mission in India-Pakistan 
started in 1949 and is still going on. In 
fact, once involved, Canada has nearly 
always remained in theatre well beyond 
the usual initial six-month mandate. 


6.50 We determined from our 
examination of documents related to two 
recent renewals of missions that 


information on the incremental cost of 
missions is being provided to ministers. 
Canada’s assessed share of the estimated 
cost of the mission was also indicated and 
the Department pointed out that this cost 
would have to be paid whether or not 
Canada decided to participate. The 
underlying rationale was that since a 
Canadian decision to participate would 
not entail new budgetary appropriations, 
cost was not a determining factor in the 
decision to participate. Foreign Affairs 
also pointed out that the incremental costs 
of the use of Canadian troops and 
equipment in a conflict situation are lower 
than the cost of using the services of the 
private sector or non-governmental 
organizations. 


6.51 We noted that insufficient 
analysis was provided to ministers of other 
implications for Canada or for the 
departments involved. For example, the 
impact of foregoing major scheduled 
training exercises on the operational 
readiness of Canada’s armed forces 
normally is not documented (see 

Chapter 7 on National Defence). 


6.52 We also noted that cost to Canada 
is mentioned only indirectly in the 
government’s guidelines for Canadian 
participation in UN peacekeeping, which 
were reaffirmed recently (see Exhibit 6.3). 


6.53 Furthermore, although the option 
of providing humanitarian aid was 
sometimes included in documents to 
ministers, it was not clear whether options 
other than the purely military option, such 
as different mixes of military and civilian 
components, are normally considered. 


6.54 The extent to which the 
guidelines can be observed in any given 
situation is usually not referred to in 
documents to ministers. Officials 
explained to us that other factors, such as 
a pressing need to respond to severe 
emergency needs based on humanitarian 
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considerations, often preclude observance 
of the guidelines. Thus it appears that 
these guidelines represent a desirable list 
of conditions in an ideal situation rather 
than a realistic basis for deciding whether 
or not to participate. National Defence has 
pointed out that the guidelines are not a 
representation of policy, but rather, as the 
term suggests, merely guidelines. 


6.55 Foreign Affairs and International 
Trade has no established procedure to 
carry out evaluations or “lessons learned” 
exercises in the area of peacekeeping. 
This increases the risk of not benefiting 
fully from the experience gained from 
previous participation. We noted only one 
evaluation commissioned by the 
Department. The key lesson learned in 
this case was “as much as possible of the 
international crisis response mechanism 
must be removed from the strictly 
political dimension.” Another illustration 
of a “standback assessment” is the 
“International Response to Conflict and 
Genocide — lessons learned from the 
Rwanda experience”, which was 
conducted jointly by international donors, 
including Canada, and non-governmental 
organizations under Danish leadership. 
“Lack of Policy Coherence” is cited as a 
key problem. It is pointed out, 
“Humanitarian action cannot substitute for 
political action.” This is perhaps the most 
important finding of this assessment. We 
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believe it is also necessary to periodically 
conduct a “standback assessment” to 
determine the extent to which Canada’s 
interests have been served from a foreign 
policy perspective and what lessons have 
been learned from Canada’s experience in 
specific peacekeeping missions. 


Canadian Efforts to Reform the 
UN’s Peacekeeping Function 


More creative approaches to reform are 
required 


6.56 Foreign Affairs and International 
Trade is committed to trying to make the 
UN more effective in a number of 
significant areas. Canada is concerned 
about the UN’s capacity to initiate and 
manage peacekeeping missions, the 
soundness of its financial base and its 
ability to react rapidly to conflict 
situations. Canada has been trying to 
promote reform of the UN’s peacekeeping 
function at both the political/institutional 
level and the operational level. The results 
so far are somewhat mixed. As one 
official pointed out, “progress is made in 
millimetres.” 


6.57 We selected for examination a 
limited number of specific initiatives in 
which Canada has participated. 


6.58 At the political/institutional level 
we noted three areas of reform — 
improved consultations, strengthening of 


the Security Council. 


presence of the Canadian troops. 


settlement. 


suited to the mandate. 


L 


© there ae be a clear, achievable ee from a competent political authority like 
@ The parties to the conflict must undertake to respect a cease-fire and must accept the 
© The peacekeeping operation must undergird a process aimed at achieving a political 
@ The number of troops and the international composition of the operation must be 


@ The operation must be adequately funded and have a satisfactory logistical structure. 


Foreign Affairs and 
International Trade has 
no established 
procedure to carry out 
evaluations or 
“lessons learned” 
exercises in the area 
of peacekeeping. 


It is necessary to 
periodically conduct a 
“standback 
assessment” to 
determine the extent to 
which Canada’s 
interests have been 
served from a foreign 
policy perspective. 


Exhibit 6.3 
Government Guidelines for 


Canadian Participation in UN 
Peacekeeping 


Source: Minister of Foreign 


Affairs’ speech, 25 January 1994 
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Canada has been 
trying to promote 
reform of the UN’s 
peacekeeping function 
at both the 
political/institutional 
level and the 
operational level. 


the rapid reaction capability, and changes 
to the scale of assessment. 


6.59 Improved consultations. Canada 
has been promoting better consultation 
between the Security Council members 
and troop-contributing countries. 


6.60 The UN Security Council 
authorizes the establishment of a 
peacekeeping operation by means of a 
resolution passed by the Council. 
Subsequent resolutions can modify, or add 
to, the role of the peacekeeping force. 
However, the problem is that the Council 
has been passing resolutions without 
sufficiently taking into account the 
practical concerns of the nations that 
contribute troops. This was causing 
practical problems that could have been 
avoided had there been a requirement for 
better consultations beforehand. 


6.61 At the insistence of Canada and 
others, the informal practice of dialogue 
between the Security Council and 
troop-contributing countries has been 
formalized. 


6.62 To this end, in the spring of 1994, 
major troop-contributing countries met in 
Ottawa, at Canada’s initiative. This 
meeting led to the creation of three 
working groups on political direction, 
command and control and training of 
peacekeeping personnel. Three papers 
were produced and sent to the UN 
Secretary General in November 1994. 


6.63 Subsequently, a Security Council 
presidential statement was issued 
specifying that meetings should be held 
“as a matter of course” among members 
of the Council, troop-contributing 
countries and the Secretariat to facilitate 
the exchange of information and views “in 
good time” before the Council makes 
decisions about the mandate of a 
particular peacekeeping mission. 


6.64 The Department continues to take 
steps to improve further on these 
advances. 


6.65 Rapid reaction capability. 
Canada has submitted a broad proposal to 
the UN General Assembly based on a 
major study to strengthen the UN’s rapid 
reaction capability. The Department 
believes that, despite the appropriateness 
of this initiative, its future is unclear at 
this time, given the fiscal and other 
constraints of the UN. 


6.66 In recent years, the UN has had 
difficulty in responding quickly and 
effectively to emerging crisis situations. 
The critical lesson of the Rwanda 
experience, for example, is that modest 
but timely intervention can make the 
difference between stabilizing a situation 
and having it spiral out of control. These 
considerations, in the Department’s view, 
argue strongly for a more effective UN 
rapid reaction capability. 


6.67 In response to the highly 
publicized genocidal massacre that took 
place in Rwanda, the Minister of Foreign 
Affairs highlighted the need for an 
in-depth review of options to strengthen 
the UN’s rapid reaction capability in times 
of crisis. Over the subsequent year, the 
Department and National Defence, in 
collaboration with other government 
agencies such as the RCMP and CIDA, 
conducted a major study. The study led to 
a report entitled, “Jowards a Rapid 
Reaction Capability for the United 
Nations”, which was submitted in 
September 1995 to the fiftieth session of 
the UN General Assembly. 


6.68 The report’s central objective is 
“to recommend changes at all levels of the 
UN system that would give the UN an 
enhanced capability to respond rapidly to 
crisis situations” (see Appendix 2). 
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6.69 Canada is working to gain 
support for the main recommendations of 
the report. In view of the current fiscal 
situation, and the longer-term implications 
for cost and reallocation of resources, 
approval may not necessarily be viewed 
sympathetically by many members. The 
Department is hopeful that at least one of 
the recommendations — the establishment 
of an operational-level headquarters — 
may still be possible in a reduced version 
in the short term. This aspect may be 
implemented by contributions from 
individual countries. The Department 
hopes that these headquarters will prove to 
be useful and that this success will help 
persuade member nations to fund the 
headquarters as part of the UN. 


6.70 Changes to scale of assessment. 
Canada has taken the position that the 
scale of assessment must be simplified 
and made more equitable. The scale of 
assessment is the formula used by the UN 
to calculate the assessed shares of 
members, for both the regular budget and 
the peacekeeping budget. Assessed shares 
are supposed to be based on the principle 
of “capacity to pay”. The practice has 
been to use national income adjusted for 
certain factors. However, several 
distortions have developed over time as a 
result of adjustments made to national 
income figures. The assessed shares of 
many members, based on the current 
scale, no longer correspond to their true 
capacity to pay. For example, the 
Department indicated that Japan’s share 
appears to be lower than its capacity to 
pay and Russia’s is higher, based on their 
reported national income figures. 


6.71 Canada, like certain other 
Western nations, may be paying more than 
its fair share if national income is 
considered the sole criterion for capacity 
to pay (see Appendix 3). We have 
encouraged Foreign Affairs and 
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International Trade to calculate what the 
impact would be on its share if the 
position it has taken on this matter at the 
UN were adopted. 


6.72 The Department believes that 
while Canada and certain other members 
support change to the scale of assessment, 
it is unlikely that General Assembly 
acceptance will be obtained in the near 
future. The political will to resolve this 
issue is not yet evident. 


6.73 Canada, together with other 
members, has been active in several 
working groups on this issue. There are 
various proposals for removing some of 
the distortions. A proposal made by the 
European Union was put forward recently. 
If it is accepted, Canada’s share will 
decrease to 2.9 percent from the present 
3.1 percent, according to the Department, 
and could conceivably result in savings of 
over $9 million annually in assessed share, 
based on the UN’s current annual 
expenditures for general and peacekeeping 
purposes. A more equitable formula might 
persuade members to pay their assessed 
shares and possibly catch up on their 
arrears. The resulting improvement in the 
UN’s peacekeeping budget would enable 
the UN to play a more proactive role in 
peacekeeping, particularly in providing 
appropriate resources to missions. 


6.74 There was hope for considerable 
support for change. The issue has been 
scrutinized by a number of advisory 
committees at UN headquarters. The 
recommendations of these committees 
will be considered by the General 
Assembly some time in 1996. Since this 
issue has fundamental financial 
significance, it will require the consensus 
of the General Assembly to be accepted. 
However, departmental officials believe 
that this support may not be possible at 
this time. 


The Department 
believes that while 
Canada and certain 
other members 
support change to the 
scale of assessment, 
the political will to 
resolve this issue is 
not yet evident. 


The assessed shares 
of many members, 
based on the current 
scale, no longer 
correspond to their 
true capacity to pay. 
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The overall cost to 
Canada of its 
participation in 
peacekeeping is 
unclear. 


The Department’s view 
is that, given Canada’s 
commitment to the UN, 
Canada needs to 
continue its efforts to 
bring about reform 
despite the slow 
progress. 


6.75 Operational level. Canada is 
promoting, among other initiatives, a 
revised budgetary and reimbursement 
system for peacekeeping operations (see 
Appendix 4). In addition, several 
Canadian officials are on loan to the UN 
Department of Peacekeeping Operations 
to assist in streamlining administrative 
and management systems. 


6.76 In summary, the above examples 
show that officials have been active in a 
variety of ways to assist the reform effort. 
Canadian efforts are appreciated by UN 
management; however, despite these 
efforts, we believe that reform at the 
political/institutional levels will be 
difficult to achieve in the short term. Such 
reform requires a consensus that is not 
easily obtained and usually requires 
lengthy negotiations among members 
whose interests are often fundamentally 
different. 


6.77 The Department indicated that 
the UN system is the only internationally 
accepted mechanism that, in the right 
circumstances, has the potential to 
contribute to peace and security. 
Therefore, its view is that, given Canada’s 
commitment to the UN, Canada needs to 
continue its efforts to bring about reform 
despite the slow progress. We agree with 
the Department, but we believe that it 
would be appropriate for the Department 
to do an assessment of its efforts to date 
and, if necessary, consider other 
approaches to reform. This observation is 
consistent with what we said in our 1991 
Report Chapter 12, “Membership 
Payments to International Organizations”: 
develop a more innovative approach to 
effecting reform in areas that have not 
changed substantially in spite of sustained 
efforts by Canada over many years, both 
alone and with other countries. The 
chapter referred to options that could be 


explored in the area of administrative 
reform. 


6.78 Canada could contribute, in 
partnership with other countries, to 
bringing about much-needed reform of the 
UN. A more proactive and creative 
approach may be necessary if Canada 
wishes to make a difference. 


The Cost of UN Peacekeeping to 
Canada 


Peacekeeping costs have risen sharply 
in recent years 


6.79 The overall cost to Canada of its 
participation in peacekeeping is unclear. 
Costs are incurred by several government 
departments/agencies. Some are incurred 
at headquarters, others in the field. Some 
are easily quantifiable, others are not. 
Some are reimbursable by the UN, others 
are not. Furthermore, whether it is more 
appropriate to use incremental or full costs 
of National Defence’s participation 
depends on what view is taken of its role 
in peacekeeping. 


6.80 Peacekeeping is not listed as a 
government program, nor is it identified 
as a distinct line of activity in either 
Foreign Affairs and International Trade or 
National Defence. Thus there is no 
separate budget for peacekeeping and no 
report is provided on its cost to Canada. 
Foreign Affairs and International Trade 
views participation in UN peacekeeping as 
one of the ways in which it supports the 
UN and at the same time serves Canada’s 
security objectives. National Defence 
views peacekeeping as a task that its 
trained multipurpose force is required to 
undertake from time to time at the specific 
request of the government. 


6.81 To examine peacekeeping costs, 
we reviewed expenditures incurred 
directly in support of a specific mission 
and expenditures incurred to support UN 
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peacekeeping activities generally. We 
found that financial data are compiled by 
different departments on their contribution 
to specific missions. Data on 
non-mission-specific costs have not been 
compiled and aggregated by any 
department. 


6.82 We are reporting only on 
mission-specific costs. To do this, we 
brought together cost information from a 
number of sources. Mission-specific costs 


include: 


¢ Canada’s assessed share paid by 
Foreign Affairs and International Trade; 


e incremental National Defence costs; 
and 


¢ costs incurred by other government 
departments/agencies. 
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6.83 Exhibit 6.4 provides an overview 
of the mission-specific financial costs to 
Canada for the period of financial year 
(FY) 1986 to FY 1996. As shown, these 
costs have risen sharply in recent years, 
from $47 million in 1991-92 to an 
estimated $240 million in 1995-96, an 
increase of 410 percent. On a cumulative 
basis, these costs total an estimated 

$1.1 billion from 1991-92 to 1995-96 
inclusive (composed of $475 million in 
UN peacekeeping assessments, 

$832 million in National Defence 
incremental costs less $236 million 
recovered by National Defence from the 
UN so far, and $27 million in costs 
incurred by other government 
departments). 


6.84 
peacekeeping to Canada included National 
Defence’s full costs (as opposed to only its 


If the cumulative cost of 


LOO OOD SOS 1994 1995 1996 


Fiscal Year 


Note 1: A portion of 1995-96 costs are predictions and are subject to change. 


NATO (IFOR) and Voluntary 


Mission-specific costs 
have risen sharply in 
recent years. 


Exhibit 6.4 


Canadian Peacekeeping 
Costs ($ millions) 


Haiti — to be finalized 


Other government 
department costs 


DND’s net incremental 
costs* 


UN assessed 
peacekeeping shares 


* DND’s Net Incremental Costs = Incremental DND Costs - UN Reimbursements 


Source: Department of National Defence, Department of Foreign Affairs and International Trade, and other government departments. 


Report of the Auditor General of Canada — May 1996 


6-21 


nL EEE 


Peacekeeping — Foreign Affairs and International Trade Canada 


Canada is obliged to 
pay its assessed share 
of each peacekeeping 
mission whether or not 
it decides to 
participate in the 
mission. 
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incremental costs), as calculated by 
National Defence, the number reported for 
the same period (1991-92 to 1995-96) is 
estimated at $3 billion. The full cost 
approach is not used by National Defence 
for decision-making purposes. We were 
informed by National Defence that it 
provides information on full costs in order 
to give full visibility to Parliament and the 
public on its peacekeeping activities. 


6.85 In addition to the above-noted 
costs, Canada is contributing about 1,000 
troops to the new 60,000-strong NATO-led 
peace implementation force (IFOR) in 
Bosnia. IFOR has taken over from 
UNPROFOR to oversee the 
implementation of the Dayton peace 
agreement. Canada’s assessed share of 
NATO common costs is being estimated 
for 1995-96 by National Defence. In 
addition, Canada will have to bear the full 
burden of its participation, which it would 
not have had to do if the mission had 
remained a UN mission. There are also 
costs associated with Canada’s decision in 
March 1996 to participate voluntarily in 
Haiti. These costs had not been finalized 
at the time of our audit; however, they are 
identified in Exhibit 6.4 as additional 
costs of peacekeeping to Canada for 
which the full magnitude has yet to be 
determined. 


6.86 Despite the recent sharp increases 
in reported peacekeeping costs to both 
departments, these costs still represent a 
modest proportion of their respective 
budgets. In 1994-95, Canada’s assessed 
share of $135 million represented about 9 
percent of the budget of Foreign Affairs 
and International Trade, and incremental 
National Defence costs of $211 million 
represented about 1.8 percent of the 
budget of National Defence. 


6.87 Canada’s assessed share. As a 
member of the UN, Canada is obliged to 
pay its assessed share of each 


peacekeeping mission authorized by the 
Security Council whether or not it decides 
to participate in the mission. This amount 
has increased progressively in recent years 
as a result of the significant increase in the 
number and size of UN missions. It went 
up from $32 million in 1991-92 to 

$79 million in 1992-93, to $128 million in 
1993-94, to $135 million in 1994-95 and 
down to an estimated $100 million in 
1995-96. 


6.88 
their dues within 30 days. Canada has 


UN rules require members to pay 


generally paid in full and on time; in fact, 
sometimes it has paid earlier than 
required. Internal auditors of Foreign 
Affairs and International Trade noted in 
late 1994 that early payments of dues to 
the UN and its specialized agencies had 
cost Canada $325,000 in imputed interest 
costs during 1993-94. The Department 
informed us that it has since reconsidered 
the benefits of paying before the due date 
and discontinued the practice. 


6.89 As mentioned earlier in the 
chapter, many members are seriously 
behind in paying their dues, for both the 
regular and the peacekeeping budgets. 
One result is that the UN owes an 
estimated US $1 billion to troop 
contributors, including significant sums to 
Canada. The Department believes that it 
does not have the right to unilaterally 
offset the amounts owed to Canada against 
assessed shares paid to the UN. 


6.90 Incremental National Defence 
costs. National Defence (DND) calculates 
“full DND cost” and “incremental DND 
cost” for each mission in which it is 
involved. “Full DND cost” is defined as 
the cost of the operation to National 
Defence. Included in this cost are civilian 
and military wages, overtime and 
allowances, full costs for petroleum, oil, 
lubricants, spares, contracted repair and 
overhaul, as well as attrition (including 


Report of the Auditor General of Canada — May 1996 


depreciation) costs for all equipment 
involved. 


6.91 “Incremental DND cost” is 
derived from “full DND cost” by 
excluding fixed costs such as salaries, 
normal operating costs and depreciation 
on equipment, and other overhead costs 
that would have been incurred by National 
Defence whether it participated or not. In 
short, incremental DND costs are directly 
attributable to National Defence for 
sending troops and equipment and 
providing goods and services specifically 
requested by the UN. “Incremental DND 
cost” is used by National Defence to 
determine “affordability” for a given 
mission. 


6.92 Incremental DND costs can vary 
significantly from mission to mission in 
their magnitude and composition. The 
examples of two recent missions, 
UNPROFOR in the former Yugoslavia and 
UNAMIR in Rwanda, illustrate this point 
(see Appendix 5). 


6.93 Although the loss of life or a 
permanent disability cannot be quantified 
in human terms, there are direct costs 
associated with Canadian troops killed or 
disabled while on peacekeeping duty that 
have not been reported as a cost of 
peacekeeping. National Defence points 
out that the costs of death and disability 
are “centrally funded and administered 
outside of DND”. The fact remains that 
death and disability represent costs of 
peacekeeping that are not provided 
separately in any department’s Estimates. 


6.94 Incremental DND costs have 
risen sharply in recent years, for the same 
reason that Canada’s assessed share has 
gone up: the increase in the number and 
size of missions. These costs were 

$ 30 million in 1991-92, $158 million in 
1992-93, $231 million in 1993-94, 

$211 million in 1994-95 and an estimated 
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$202 million in 1995-96. These costs 
represent the greatest component of the 
overall cost to Canada of peacekeeping. 


6.95 Costs incurred by other 
government departments and agencies. 
Other departments and agencies, such as 
CIDA, RCMP, Elections Canada and 
Revenue Canada, may get involved from 
time to time in specific missions. This is 
particularly true of the more recent ones in 
which there has been a much greater need 
for intervention in non-military areas, 
such as the maintenance of civil law and 
order and monitoring of elections. For 
example, under an agreement between 
Foreign Affairs and International Trade 
and CIDA, the latter is using Official 
Development Assistance funds for the cost 
of 100 Canadian police officers serving 
with the UN mission in Haiti. Other 
examples are Elections Canada officials 
monitoring elections in Namibia and 
Revenue Canada officials monitoring 
sanctions against Serbia. However, these 
costs are much smaller compared with 
Canada’s assessed share or the 
incremental costs of National Defence 
(see Exhibit 6.4). 


6.96 Non-mission-specific costs. 
Non-mission-specific costs are costs that 
are incurred for peacekeeping-related 
activities but that are not attributable to a 
specific mission. The subsidy of 

$10 million over five years to the Pearson 
Peacekeeping Centre, equally shared by 
Foreign Affairs and International Trade 
and National Defence, is one such cost. 
Other examples are the incremental costs 
of foreign allowances, accommodation, 
travel, meals, facilities and other costs 
associated with Canadian officials on loan 
to the UN’s Department of Peacekeeping 
Operations; development of proposals 
designed to improve UN peacekeeping 
operations; organizing conferences and 
meetings related to specific proposals; and 


Incremental DND costs 
are directly 
attributable to National 
Defence for sending 
troops and equipment 
and providing goods 
and services 
specifically requested 
by the UN. 


Direct costs 
associated with 
Canadian troops killed 
or disabled while on 
peacekeeping duty 
have not been 
reported as a cost of 
peacekeeping. 
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CIDA provides 
significant amounts of 
humanitarian 
assistance through UN 
agencies. The 
expanded definition of 
UN peacekeeping 
could arguably include 
such costs. 


administration involved in obtaining 
reimbursements from the UN. In our view, 
these costs should be reported to 
Parliament as peacekeeping costs in any 
compilation of the overall cost of 
peacekeeping to Canada. 


6.97 CIDA humanitarian assistance 
is not viewed as a direct peacekeeping 
cost. It is also important to note that 
CIDA provides significant amounts of 
humanitarian assistance through UN 
agencies and international and Canadian 
non-governmental organizations. CIDA 
has pointed out that its interventions are 
carried out in pursuit of its own 
developmental and humanitarian mandate 
and not primarily because a UN mission is 
active in the conflict area. We have 
therefore not included CIDA costs in our 
analysis, although it seems to us that the 
expanded definition of UN peacekeeping 
set out in paragraphs 6.17 to 6.23 could 
arguably include such costs. 


Canada’s Administration of 
Reimbursements from the UN 


Improvements are needed 


6.98 We have three main concerns in 
the area of financial administration. 
First, as at 31 March 1996, about 

$92 million was receivable from the UN; 
this will require continued diligence to 
collect. Second, Canada has not billed the 
UN for death and disability payments. 
Third, National Defence’s accounts 
receivable were understated by at least 

$36 million as at 31 March 1995. 


6.99 UN missions are funded by all 
members. The UN uses peacekeeping 
funds collected from its members, based 
upon the scale of assessment, to cover the 
mission operating expenses and to 
reimburse certain eligible costs to troop 
contributors. In theory, costs incurred by 
troop contributors are shared by all 
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member states. In practice, however, 
because the UN uses standard rates of 
reimbursement for troops and equipment 
depreciation, reimbursement received by 
“higher-cost” countries, such as Canada, is 
usually not sufficient to cover the costs 
incurred by them, and may be more than 
sufficient to cover the costs incurred by 
“lower-cost” countries. 


6.100 What does the UN reimburse? 
The UN reimburses troop-contributing 
nations for the following: 


¢ troop costs at a rate of US $988 per 
person-month, plus $291 per 
person-month for specialists, and up to 
$70 per person-month for other factors; 


e Letters of Assist (contracts for goods 
and services with the UN); 


e consumable supplies and equipment 
depreciation; and 


e death and disability while on UN 
duty. 


6.101 In addition, the UN pays a per 
diem food and lodging allowance directly 
to military observers, non-military 
personnel, and other staff for which the 
UN is responsible, such as RCMP officers 
serving in Haiti (US $123 tax-free per 
person for the first 30 days, and $87 per 
day thereafter). 


6.102 Administrative requirements to 
obtain reimbursements from the United 
Nations. Reimbursements are made to 
troop contributors by the UN in 
accordance with administrative guidelines 
produced for each peacekeeping mission. 
These guidelines vary little from mission 
to mission.. 


e Troops costs are paid automatically 
by the UN based upon troop strength 
reports collected by UN officials in 
theatre. 
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e Letters of Assist are reimbursed on 
the basis of invoices supported by detailed 
documentation submitted to the UN. 


e Consumable supplies and equipment 
depreciation are reimbursed on the basis 
of a detailed inventory of eligible items 
brought to the theatre by the contingent 
(in-survey), and an inventory of eligible 
items taken out at the end of a mission 
(out-survey). 


6.103 How does Canada administer 
its peacekeeping transactions with the 
UN? Foreign Affairs and International 
Trade views its role primarily as that of 
paying Canada’s UN-assessed share in full 
and on time for each peacekeeping 
mission. The Department informed us that 
it does not assume any accountability for 
the administration of reimbursements 
from the UN. In its opinion, National 
Defence is fully responsible and is 
expected to fulfill this function with 
support from Canada’s Permanent Mission 
to the UN as and when needed. National 
Defence staff acknowledge that a more 
proactive role of Canada’s Permanent 
Mission to the UN is crucial to the 
collection function. 


6.104 Canada’s Permanent Mission to 
the UN acts simply as a receiving and 
forwarding agent between the billing and 
collection group in National Defence and 
the appropriate section in the UN. Its role 
in follow-up, collection or accounting 
depends upon instructions from National 
Defence. We were informed that the 
permanent missions of certain other 
member states are actively involved in the 
administration and reimbursement 
process. 


6.105 As of 1 April 1995, we were 
informed that National Defence assigned 
four staff fully to the UN reimbursement 
function whereas in the past, staff were 
not dedicated to this function. The same 
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staff are also required to travel 
periodically to UN headquarters to consult 
with UN officials, and frequently to the 
missions in theatre to gather 
documentation needed to support invoices. 


6.106 Where do reimbursements from 
the UN go? Canada has recovered 

$277 million from the UN during the last 
twelve years, with the bulk of it in the last 
four to five years (see Exhibit 6.5). Until 
1993-94, most revenue recovered from 
the UN was credited to the Consolidated 
Revenue Fund. In the last two years, 
National Defence has retained a total of 
$36.4 million to offset its increasing 
incremental costs, related primarily to the 


Canada has recovered 
$277 million from the 
UN during the last 


increasing use of Letters of Assist by the 12 years. 
UN. 
6.107 Incremental DND costs are 


absorbed by National Defence within its 
budget. It has not received any extra funds 
for peacekeeping activities other than the 
$10 million that we were told was added 
to its reference level in 1976. National 
Defence is currently negotiating with 
Treasury Board for approval to retain all 
future recoveries from the UN to cover its 
incremental costs, because the amounts 


Exhibit 6.5 


Reimbursements Received by 
Canada from the UN ($ millions) 


: Revenues returned 
Fiscal Total UN to Consolidated Revenue 
Year Revenue Received Revenue Fund kept by DND 
= = 
1985-86 to 
1990-91 40.4 40.4 O 
1991-92 153 ioe pe 
1992-93 11.5 11.0 S 
1993-94 51.7 ele: S) 
1994-95 86.1 79.7 6.4 
1995—96* 79 41.9 30.0 | 
a as T 
Total 276.9 239.5 37.4 


* partial figures up to 28 February 1996 


Source: Department of National Defence 
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involved have become much larger. We 
were informed that Treasury Board is 
likely to approve the Department's 
request. We would encourage National 
Defence to also seek a change in its vote 
wording to make the authority more 
specific for retaining recoveries from the 
UN. 


6.108 Amounts collectable from the 
UN. In order of priority, the United 
Nations reimburses troop costs, Letters of 
Assist (LOAs), consumable supplies and 
equipment depreciation. The problem of 
slow recoveries is related primarily to the 
latter three costs. An LOA claim cannot 
be certified unless it is properly 
documented and supported with evidence. 
Often the evidence required has to be 
obtained from the mission in theatre. It is 
therefore important that field staff be 
familiar with UN documentation 
requirements and that prearranged 
procedures be in force to collect such 
documentation. According to UN officials 
responsible for processing claims, those 
countries that submit their claims in a 
complete and timely manner are likely to 
get reimbursed first. Meeting the UN’s 
procedural requirements is often difficult 
and time-consuming. As experience has 
shown and the following examples 
illustrate, it is very difficult and 
sometimes impossible to get the evidence 
after the fact: 


e compensation under a Letter of 
Assist for 51 vehicles (trucks valued at 
$232,000) sold to the UN has been 
delayed because of missing 
documentation; and 


e a total of 131 vehicles (various types 
valued at $1.9 million) taken by National 
Defence since 1992 on the UNPROFOR 
mission were lost or destroyed. 
Documentation to obtain reimbursement 
was sent to the UN in January 1996. 


6.109 Another example of 
compensation collectable from the UN is 
for death and disability suffered while on 
UN peacekeeping duties. The UN 
compensates member nations for 
payments awarded by them to the soldier 
or soldier’s family. However, Canada has 
not yet billed the UN for death and 
disability compensation for personnel 
other than military observers. We were 
informed by National Defence that, during 
the past few years, it has been working to 
address this issue of compensation. 
Officials are currently working to 
conclude 14 case files, representing claims 
for deaths incurred over the past 

five years, and to prepare them for 
submission to the UN for compensation. 
The precise amount recoverable has not 
been ascertained and will be finalized 
only through negotiations with UN 
officials. With respect to the incidents of 
death and disability prior to 1991, we 
were informed by National Defence that 
the amount of lost revenue could amount 
to “millions of dollars”. Canada, like 
many other nations, appears to be in 
arrears with respect to pursuing 
compensation in this area. Certain other 
countries, although not all, have been 
much more active in billing the UN for 
death and disability (see Exhibit 6.6). Our 
concern is that any past inattention to this 
important aspect of financial management 
not be repeated in future years. In our 
opinion, Canada needs to continue to 
actively pursue the submission and 
settlement of claims relating to death and 
disability. 


6.110 At the time of our audit, amounts 
receivable by Canada from the UN for 
troop costs, Letters of Assist, consumables 
and equipment depreciation up to 

31 March 1996 were about $92 million. 
This includes $17 million for Cyprus that 
National Defence believes can be 
collected only if contributions are 
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received by the UN for this voluntary 
mission. The $92 million does not include 
unbilled amounts for death and disability. 


6.111 Billings. We were informed by 
National Defence that invoices were 
prepared in accordance with departmental 
and UN procedures at the time of 
submission. However, the changing 
policies and procedures of the UN, most 
applied retroactively, have resulted in a 
requirement to provide additional 
information prior to the UN reviewing the 
invoices for payment. In other instances, 
Canadian and UN policies and procedures 
are in conflict and further negotiations are 
therefore required to settle reimbursable 
amounts. National Defence claims that it 
actively pursues settlement of all 
outstanding invoices. It acknowledges that 
it takes time to get its bills to the UN in 
some instances, but points out that it has 
only a small team handling the 
time-demanding task of costing and 
preparing invoices to meet UN 
requirements. It also emphasizes that the 
UN should not be viewed as, and 
compared with, a private sector 
organization. In National Defence’s view, 
the UN isa huge bureaucracy with its own 
administrative rules and multiple levels of 
approval. Practices of billing and 
collection that are normal in the business 
world are not necessarily fully applicable 
to the UN. 


6.112 Recoveries from the UN are also 
a function of the availability of funds. The 
organization’s serious cash shortage 
impedes recoveries, despite best efforts, 
simply because the UN does not always 
have the cash to make payments on a 
timely basis. Recently, peacekeeping 
funds have been used to meet regular 
budget expenses. The amount of funds 
available may influence the UN’s decision 
on when, how much, and whom to 
reimburse. We were informed that a 
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higher priority may be given to “less 
wealthy” countries that depend heavily 
upon reimbursements. The result is that 
even if invoices are submitted on time, 
sometimes reimbursement may still not be 
possible because there are no funds left or 
the funds available are dependent upon 
voluntary contributions from other 
member states. Over $17 million shown as 
a receivable at National Defence, owed to 
Canada for its participation in Cyprus, 
remains uncollected. It will probably 
never be collected because funds are 
unavailable in that account, as a result of a 
lack of voluntary contributions. 


6.113 Understatement of accounts 
receivable in National Defence’s 
accounts. Canada accounts for all non-tax 
revenues based on the accrual system of 
accounting, in which such revenue is 
recognized and an account receivable set 
up as soon as a good or service has been 
provided by Canada. National Defence’s 
accounting system, however, was designed 
on a basis of accounting in which revenue 
from the UN is recognized only when an 
invoice has been sent or a payment has 
been received, whichever is sooner. Thus, 
amounts due to Canada for troop costs, 
consumable supplies and equipment 
depreciation will not show as receivables 
because no invoice is required to be sent. 


Recoveries from the 
UN are also a function 
of the availability of 
funds. 


At the time of our 
audit, amounts 
receivable by Canada 
from the UN up to 

31 March 1996 were 
about $92 million. This 
total does not include 
unbilled amounts for 
death and disability. 


Exhibit 6.6 


Death and Disability Claims 
Paid by the UN to Certain 
Other Countries (period of 
1992 to 15 September 1995) 


Total 
# Claims Amount Claimed Average 
Processed US $ (millions) US $ 
T coe: 
Death 159 16.5 104,000 
Disability 322 20.8 65,000 
Total 481 S13 


Source: United Nations 
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There was an 
understatement of at 
least $36 million in the 
accounts receivable 
reported by National 
Defence as at 

31 March 1995. 


There is need for a 
more direct role in 
oversight and 
monitoring by senior 
management. 


Improvements are 
possible in the 
administration of 
reimbursements from 
the UN. 


Nor will revenue be recognized on a 
timely basis because the UN is unable to 
make reimbursements as a result of cash 
shortages. Only amounts owing to Canada 
for Letters of Assist are reflected in the 
accounts because they are based on actual 
invoices sent to the UN. 


6.114 There was an understatement of 
at least $36 million in the accounts 
receivable reported by National Defence 
as at 31 March 1995. There is a need to 
introduce full accrual accounting for such 
revenue items in the accounts of National 
Defence. We were informed that this 
situation is being corrected for the fiscal 
year ending 31 March 1996. 


6.115 Improvements will need 
continued senior management attention 
in both departments. We believe 
improvements are possible in the 
administration of reimbursements from 
the UN. At the operational level, more 
attention by personnel in theatre to the UN 
reimbursement process and requisite 
documentation for claims will help. 
Furthermore, more involvement of 
Canada’s Mission to the UN in the 
administration of reimbursements may be 
necessary. 


6.116 Initiatives taken by National 
Defence. Many improvements have been 
made over several years. For example, the 
number of staff involved in this task has 
been increased; special teams have been 
sent to missions in theatre to complete 
in/out surveys; training on administrative 
requirements for UN reimbursement has 
been provided; and Canada has made a 
proposal for a revised budgetary and 
reimbursement system for peacekeeping 
operations (see Appendix 4). The last 
initiative is an example of one of a 
number taken by Foreign Affairs and 
National Defence to reform the UN’s 
peacekeeping function (see 


paragraphs 6.55 to 6.77 for other 
examples). 


6.117 At present, Foreign Affairs and 
International Trade and the Permanent 
Mission assume no direct responsibility 
for collecting reimbursements from the 
UN. The Department believes that this 
area is the responsibility of National 
Defence. However, the External Affairs 
Act assigns to the Department the 
responsibility for the conduct of all 
diplomatic relations with international 
organizations such as the UN on behalf of 
Canada, and Canada’s Ambassador to the 
UN reports to the government through the 
Department. 


6.118 In our opinion, there is need for a 
more direct role in oversight and 
monitoring by senior management at 
Canada’s Mission in New York. But in 
order to do this and because the Mission 
follows instructions from Ottawa, senior 
management of both Foreign Affairs and 
National Defence have to give this area 
higher priority. The Department’s 
accountability for administration needs to 
be clarified. 


Conclusion and 
Recommendations 


6.119 The post-Cold War period has 
brought about many fundamental changes. 
Specifically, in the area of security, the 
origin, nature and complexity of conflicts 
have changed significantly. In response, 
the nature of peacekeeping and the 
demands that are placed on the UN have 
also changed in many ways. Peacekeeping 
mandates and missions are more 
numerous, more complex and more costly. 
While peacekeeping in the past was 
largely a matter of monitoring peace 
agreements, today’s peacekeeping is often 
expected to address concurrently a vast 
range of problems: military, political, 
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humanitarian, health, migration and 
resettlement, civil law and order, and 
human rights. The UN is not equipped, 
nor does it have the resources, to deal with 
all of them. Furthermore, the UN’s own 
record in peacekeeping has come under 
close scrutiny; traditional troop 
contributors are reducing their armed 
forces; major donors are tackling serious 
domestic fiscal problems; and the UN 
itself is faced, for a variety of reasons, 
with the worst financial crisis in its 
history. 


6.120 Canada has been a long-standing 
supporter of the UN and its peacekeeping 
function. It continues to consider the UN 
as fundamental to meeting its foreign 
policy objectives. Canada’s past record in 
UN peacekeeping has been laudable and 
participation in UN peacekeeping enjoys 
substantial public support. We believe, 
however, that given the fiscal and other 
realities in Canada and at the UN, there is 
a call for greater transparency and 
openness to better inform Parliament and 
the people of Canada on all significant 
aspects of peacekeeping. 


6.121 Foreign Affairs and 
International Trade Canada should: 


¢ conduct in co-operation with other 
government departments and 
non-governmental organizations a 
“standback assessment” to determine 
what lessons have been learned; 


¢ consider other ways, if any, in 
which Canada could participate in 
peacekeeping to meet Canada’s 
objectives, given the fiscal and other 
realities in Canada and at the UN; and 


¢ ensure more public articulation of 
the nature and extent of participation in 
specific missions and the potential for 
obtaining the benefits Canada wishes to 
derive. 
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6.122 Foreign Affairs and 
International Trade, in consultation 
with National Defence, the Privy 
Council Office and the Treasury Board 
Secretariat, as appropriate, should seek 
to: 


¢ obtain designation of a lead 
department for reporting annually to 
Parliament on all important aspects of 
Canadian participation in 
peacekeeping, and specifically 
information on: 


— all significant costs, 
implications, and benefits to 
Canada; 


— additional ways, if any, that 
were considered for 
participation in UN 
peacekeeping; and 


— efforts made toward UN reform 
related to peacekeeping, and 
the results achieved; and 


e clarify the accountability of 
Canada’s Permanent Mission to the UN 
for administration related to 
peacekeeping reimbursements. 


Foreign Affairs and International Trade 
Canada’s response: Foreign Affairs and 
International Trade Canada concurs in 
general terms with this chapter of the 
Auditor General’s Report. 


The chapter recognizes the importance of 
peacekeeping as an instrument of 
Canadian foreign policy and the 
contributions of Canada as a respected 
member of the UN. It also notes that the 
costs of peacekeeping have risen in recent 
years, due to an increase in the number, 
size and complexity of UN peacekeeping 
missions. The Department notes, however, 
that the costs of UN peacekeeping are 
likely to decline substantially in coming 
months and years, due to a drop in the 
number of new missions, the wind-up of 
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some of the larger operations and the 
smaller size of the UN's current 
operations. It should be noted that 
Canada pays its assessed shares for these 
operations to the UN as a matter of legal 
obligation irrespective of whether we 
contribute personnel or equipment. 


Foreign Affairs and International Trade 
Canada sees the question of 
reimbursement of funds from the UN to the 
Government of Canada as an issue wholly 
within the purview of the Department of 
National Defence. National Defence has 
the technical expertise to calculate the 
complex formulae governing 
reimbursement of funds for which it is 
responsible under Treasury Board 
regulations. This is especially important 
as some of the moneys recovered from the 
UN are retained by the Department of 
National Defence to defray the 
operational costs of peacekeeping 
missions. In the view of Foreign Affairs 
and International Trade, the lines of 
financial responsibility on this issue within 
the Government of Canada are clear. 


With respect to information on Canada’s 
commitments to peacekeeping, there is no 
single source of information on all of 
Canada’s engagements. Moreover, 
because a number of departments and 
agencies are involved in modern 
peacekeeping operations, it would be 
inappropriate to designate a single “lead 
agency” in this field. Information on 
peacekeeping could be reviewed in 
conjunction with other departments and 
agencies to ascertain whether additional 
information could be made available in a 
consolidated format. 


Department of National Defence’s 
response: The Department of National 


Defence acknowledges that, under the 
terms of the 1985 Order-in-Council, the 
audit team may not have had sufficient 
access to documents necessary to assess 
fully the extent of information provided to 
ministers on the military implications of 
peacekeeping. National Defence also 
supports the position of Foreign Affairs 
and International Trade that potential 
opportunities to consolidate information 
on Canada’s peacekeeping commitments 
could be explored. 


National Defence will continue to play a 
central, if not increasing, role in the 
administration of UN reimbursements. The 
difficulties inherent in the current system 
of administration are acknowledged even 
though they are for the most part 
attributable to the UN, and are largely 
outside the purview and influence of 
National Defence. Nonetheless, National 
Defence’s success in obtaining large 
reimbursements from the UN in recent 
years, and in calling for, working toward, 
and obtaining a major reform of the UN's 
reimbursement process are indicative of 
the high priority that National Defence 
gives to administration of these 
reimbursements. 


National Defence notes the observations 
in the area of death and disability claims, 
and that further work is required in this 
area. However, it must be noted that the 
problems associated with these claims 
have not jeopardized the benefits bestowed 
on Canadian Forces personnel and their 
dependents. The UN system for 
reimbursement of death and disability 
benefits is extremely complex and poses a 
significant administrative burden. It 
should also be noted that National 
Defence is pursuing improved 
arrangements with the UN in this regard. 
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% About the Audit 


Foreign Affairs and International Trade Canada is responsible for the delivery of Canada’s foreign policy, 
including advising the government on the nature and extent of its participation in UN peacekeeping. The 
Department is mandated under the External Affairs Act to conduct all diplomatic relations on behalf of 
Canada and all official communications between the Government of Canada and the United Nations. 


Objective 


The objective of our audit was to provide information to Parliament on the role of Foreign Affairs and 
International Trade Canada in the government’s decision-making process and to examine its management 
practices as they relate to Canada’s participation in peacekeeping. In this regard, our audit was extended to 
National Defence in the area of administration because the system for billings and collections is maintained 
there. 


Scope 


The scope of our audit was limited to Canadian participation in United Nations peacekeeping missions for 
which funding was provided through the UN peacekeeping budget. We focussed on the Department’s 
headquarters, Canada’s Permanent Mission to the UN, and National Defence for information related to 
National Defence’s incremental costs and the administration associated with transactions with the UN. We 
reviewed National Defence’s costs and related aspects of financial administration in this chapter because its 
role in peacekeeping is ultimately to support the delivery of Canada’s foreign policy, and payments to the UN 
are effected through Foreign Affairs and International Trade. 


Specifically, we examined the following: 


e the adequacy of information for Parliament for more informed debate: 

e the adequacy of information to ministers for decision making; 

e the extent of Canadian efforts to reform the UN’s peacekeeping function; 

e the presentation of the overall cost to Canada of peacekeeping; and 

e the quality of Canada’s financial administration of peacekeeping reimbursements from the UN. 


We identified the overall cost of peacekeeping to Canada, which was done in the interest of transparency of 
the information provided to decision makers and parliamentarians, rather than as an effort to weigh the costs 
and benefits of Canada’s participation in peacekeeping. 


Criteria 


Because the Department is responsible for formulating Canada’s foreign policy and all relations with the UN, 
we expected that it would: 


¢ provide comprehensive information to ministers on all aspects of Canada’s participation in UN 
peacekeeping; 
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e periodically conduct a “standback” assessment to determine whether Canadian foreign policy objectives 
were being met in the most cost-effective way, especially given the rapid changes in the global 
environment; 


e determine what lessons had been learned; and 


e provide stewardship for the financial transactions incurred with the UN. 


Approach 


We interviewed officials in both Foreign Affairs and International Trade and National Defence headquarters, 
at UN headquarters, at Canada’s Permanent Mission to the UN and at certain other countries’ missions to the 
UN. as well as NGO representatives, consultants and academics in Canada. We examined relevant documents 
including UN, departmental and other studies; parliamentary debates; and Part IIIs and committee reports. We 
also met with the Chair of the Standing Committee on Foreign Affairs to better understand the Committee’s 
interest in peacekeeping and its ongoing needs for information. 


We consulted with the Department’s Office of the Inspector General. The Office had not conducted an 
examination of the management of peacekeeping; accordingly, we could place only limited reliance on the 
work of the Office. 


Audit Team 


Rajesh Dayal 
Emest Glaude 
Natalie Hétu 
Manfred Kuhnapfel 
Jon Legg 

Jacques Marquis 
Daniel Thompson 


For information, please contact Vinod Sahgal, the responsible auditor. 
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Appendix 1 


Sources of Information on UN Peacekeeping Operations 


The Burgeoning Cost of UN Peacekeeping: Who Pays and Who Benefits ?, Peter Gizewski and Geoffrey Pearson, Canadian Centre 
for Global Security, November 1993. 


An Agenda for Peace: Preventive Diplomacy, Peacemaking and Peacekeeping, Report of the Secretary General, Office of the UN 
Secretary-General, June 1992. 


United Nations — U.S. Participation in Peacekeeping Operations, General Accounting Office (GAO), September 1992. 
UN Peacekeeping — Lessons Learned in Managing Recent Missions, GAO, December 1993. 

How Assessed Contributions for Peacekeeping Operations are Calculated, GAO, August 1994. 

Renewing the United Nations System, Erskine Childers and Brian Urquhart, Development Dialogue, 1994. 

Whither UN Peacekeeping ?, Mats R. Berdal, International Institute for Strategic Studies, October 1993. 

Charter of the United Nations. 

Report to the Secretary-General of the United Nations, Dick Thornburgh, March 1993. 


Comprehensive Review of the Whole Question of Peacekeeping Operations in all their Aspects, Report of the Special Committee on 
Peacekeeping Operations, June 1995. 


The Blue Helmets: A Review of United Nations Peacekeeping, United Nations, 1990. 


Final Report on the in-depth evaluation of peacekeeping operations: start-up phase, Report of the Office of Internal Oversight 
Services, UN Headquarters, May 1995S. 


Report of the Board of Auditors to the General Assembly on the Accounts of the United Nations Peacekeeping Operations for the 
biennium ended 31 December 1993. 


Report of the Board of Auditors on the audit of the Support Account for peacekeeping operations for the period ended 30 
September 1995, February 1996. 


Monthly Summary of Troop Contributions to Peacekeeping Operations, Department of Peacekeeping Operations, UN Headquarters, 
January 1996. 


Statement to the 5th Committee, Joseph E. Connor, UN Under-Secretary-General for Administration and Management, 
September 1995. 


Statement to the High-Level Open-Ended Working Group on the Financial Situation of the United Nations, Boutros Boutros Ghali, 
UN Secretary-General, June 1995 and February 1996. 


Improving the capacity of the United Nations for peacekeeping, Report of the UN Secretary-General, May 1994. 


Investigation of the relationship between humanitarian assistance and peacekeeping operations, UN Joint Inspection Unit, 
October 1995. 


Sharing responsibilities in peacekeeping: The United Nations and the regional organizations, UN Joint Inspection Unit, 
October 1995. 
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Appendix 2 


The Canadian Study on a UN Rapid Reaction Capability 


The objective of the study was to assess the validity and practicality of a variety of policy alternatives. 
Consultations were held with other governments, non-governmental organizations and intergovernmental 
institutions. Three international conferences were organized: at Cornwallis, Nova Scotia, in Quebec City and in 
Montebello, Quebec. 


The report makes 26 recommendations for the short, medium and long terms, which, in the report’s words, 
“would give the UN the rapid reaction capabilities it needs to respond quickly and effectively to crisis 
situations.” 


According to the report, its principal idea is the “vanguard concept”. “Under this concept, the UN would be able 
to assemble from member states a multi-functional force of up to 5,000 military and civilian personnel and 
rapidly deploy it under the control of an operational-level headquarters upon authorization of the Security 
Council. The operational-level headquarters, a new unit to fill a current vacuum in the UN system, would be 
responsible for the planning and advance preparations, which are crucial if rapid reaction is to work. This 
concept emphasizes the importance of making significant changes at the operational level of the UN system and 
with the troop contributors who would provide the trained, equipped forces essential to rapid reaction. Forces 
would be provided by way of enhanced standby arrangements, which the Secretariat would conclude with 
member states offering personnel to participate in peace operations. The report includes recommendations to 
enhance training, explore more efficient systems for logistics and transportation, and bolster the planning efforts 
of the entire UN system.” 
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Appendix 3 
The UN Scale of Assessment 


The scale of assessment is the method adopted by the UN in 1946 to apportion the expenses of the UN among its 
members. It is based on the principle of “capacity to pay”. The practice has been to use the national income of 
each member as a reference point. National income figures are averaged over a number of years, further adjusted 
to reflect low per capita incomes and debt levels, and upper and lower share levels are set. For peacekeeping 
operations, the five permanent members of the Security Council are assessed a greater share than their regular 
budget shares; for example, the U.S. share is 31 percent as compared with its regular budget share of 25 percent. 


Over the years, however, members have expressed dissatisfaction that the scale is too complicated, far from 
transparent, includes contradictory principles, does not reflect capacity to pay and has serious distortions. One 
example is that some states are paying less than is called for by their share of global income, and some states are 
paying more. Canada’s current ambassador to the UN said, “The current system has serious distortions and no 
longer adequately reflects members’ abilities to pay.” Other states agree. The U.S. representative in New York 
echoed the same sentiment in 1991, when he said: “Unfortunately, the reliance on capacity to pay, and the 
methodology that has gone into quantifying it, have resulted in distortions in the scale....” 


The General Assembly in 1994 authorized an “Ad Hoc Intergovernmental Working Group on the Implementation 
of the Principle of the Capacity to Pay” to study the question and make recommendations. The Working Group’s 
report, referring to the “Scale of Assessment for the Apportionment of the Expenses of the UN”, was submitted 
on 8 May 1995. According to the Department, several aspects of the scale need to be reconsidered. For example, 
the present method of debt adjustment is unsatisfactory. Another irritant is the overly long statistical averaging 
period of 7.5 years. This will be reduced to three years if the Working Group’s report is adopted. 


The Working Group also addresses the question of purchasing power parity as a possibly more appropriate 
measure of capacity to pay. The Group concluded that this concept has flaws; for example, data are not available 
on all countries, and comparability of a common basket of goods and services would be difficult. 
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Appendix 4 


A Revised Budgetary and Reimbursement System for Peacekeeping Operations 


At the present time, the mandatory process for seeking reimbursement for depreciation or loss of equipment 
starts with the completion of the in-survey. The in-survey is the methodology used by the UN to count all 
equipment and materiel brought to the mission area by the contingent. When the contingent leaves at the end of 
its mandate, an out-survey is completed and the two surveys are reconciled by the UN to determine 
reimbursement for depreciation and loss during the tour of the contingent. Unfortunately, the level of detail 
required by the UN, using standards that are appropriate for missions with little contingent-owned equipment, 
makes the process unwieldy, especially since the conflict conditions at the peacekeeping site are rarely suitable 
for this administrative procedure. 


Canada developed and received UN authority to try out a standard rate reimbursement methodology in its 
mission in Haiti. The proposals were based on taking the process used for troop cost repayments and expanding 
it to incorporate a monthly charge for equipment and services as well. The timing of this initiative coincided 
with the United Nations’ establishment of a five-phase working group to reform the methodology for 
reimbursement for contingent-owned equipment. 


The UN Working group, comprising 57 nations, expanded on the Canadian proposal and developed a 
comprehensive reform of the complete methodology for reimbursement not only for equipment but also for the 
support provided to forces in the mission area. The use of standard rates in a negotiated agreement meant that 
contributions by nations would now be clear, responsibilities for support would be laid out, and the application 
of standard rates would provide quick budget information for the mission planners. 


For Canada, the agreement would provide greater visibility of funds owed by the UN and a clear picture of the 
quantity of personnel and equipment desired by the UN. If Canada wished to enhance its participation without 
UN concurrence, the increase would be considered a voluntary contribution, a decision that could be more easily 
defined and presented to government than waiting until an in-survey is negotiated in the mission. 


The report of the Working Group to reform the major component of financing for peacekeeping operations was 
scheduled to be presented for adoption by the General Assembly in March 1996. The UN Secretariat expects the 
new system to be in effect by July 1996. At that point many of the problems associated with completing 
in-surveys, documenting inventories, accounting to the UN for loss or damage and preparing invoices will be 
eliminated. A series of inspections and verifications to ensure that the provisions of the agreement are being met 
will be introduced. Reimbursement would then be based on simple compliance reports instead of the current 
time-consuming, paperwork-intensive process. 
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Appendix 5 


Incremental DND Costs for Two 
Missions 


Reimbursements from 
the UN are excluded 


Source: Department of 
National Defence 
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FORMER YUGOSLAVIA (UNPROFOR): 
INCREMENTAL COST $379 MILLION (up to 31 March 1995) 


(DND’s full costs up to 31 March 1995 were $1 billion) 


Reserve Salaries | 
Employee 
Allowances 

Procurement 


Commercial Air i 


Water & Rations | 
Training | 
Other 


$ CDN (millions) 


RWANDA (UNAMIR): 
INCREMENTAL COST $56 MILLION (up to July 1995) 


(DND’s full costs up to July 1995 were $109 million. Included in the $56 million but 
excluded in the chart below is DND’s voluntary humanitarian contribution of a field 
hospital and airlifts for CIDA representing an incremental cost of $18 million.) 
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Note: “other” includes personnel travel, communications, ammunition, vehicle 
depreciation for usage greater than that used in normal training and various other expenses. 
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The audit work reported in this chapter was conducted in accordance with the 
legislative mandate, policies and practices of the Office of the Auditor General. 
These policies and practices embrace the standards recommended by the Public 
Sector Accounting and Auditing Board ( PSAAB ) of the Canadian Institute of 
Chartered Accountants. 
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Assistant Auditor General: David Rattray 
Responsible Auditor: Peter Kasurak 


Main Points 


rp The Canadian Forces have achieved much in their peacekeeping operations and their missions have been 
marked by individual acts of dedication and bravery. 


72 The Department of National Defence plans for peacekeeping missions in a fluid environment, often under 
severe time constraints. It has experienced gaps in its planning for deployments and rotations to missions. The 
Department, however, has improved its staff system to assess the likelihood of success of the overall mission, to 
assess the capability of the Canadian Forces to meet the requirements of a mission and to estimate the impact of a 
deployment on other commitments. 


* FS) The Land Forces have retained soldiers in their field force who otherwise would have been cut as part of 
force reductions. This accounts for 1,600 of the 3,000 extra troops allowed by the government for peacekeeping. 
Little other increase has occurred to date. Overall, even in the most highly used categories, personnel have not 
been overtasked for peacekeeping duty. There are still many practical limitations to using the Reserves for 
peacekeeping. 


7.4 Peacekeeping duty in the last few years has revealed serious problems in the Land Force’s ability to 
generate multipurpose forces: 


e The Land Force is not completing its collective training plans. Its most comprehensive training 
exercise was replaced by a much smaller exercise due to peacekeeping-related tasks. Many 
lower-level exercises planned in the last few years did not take place. 


e Significant emergency buys were needed to provide minimum levels of equipment for troops 
deployed for peacekeeping. In 1995, National Defence concluded that long-standing deficiencies in 
its armoured vehicles would not allow troops to perform assigned missions within an acceptable level 
of risk. 


¢ Departmental plans for medical support show that the Department’s planned field hospital will not 
be enough to support the Land Force’s operational plans. 


7.5 The Department sent $822 million in supplies and equipment to support the peacekeeping missions we 
studied. It recorded $67 million in downward adjustments to stock balances and $13 million of losses. During our 
audit, however, the Department could not find a significant portion of the required documentation, and the 
downward adjustments and losses remain largely unexplained. 
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Introduction 


Canadian Forces Peacekeeping 
and Its Environment 


Canada’s defence policy and 
peacekeeping 


7.6 The Canadian Forces have been 
used extensively on peacekeeping 
missions. Canada has participated in 
nearly every United Nations (UN) 
peacekeeping operation as well as other 
multilateral contingency operations, such 
as the Gulf War. Canadian Forces 
participation in peacekeeping and 
contingency operations is in support of the 
Canadian government’s foreign policy. 


Tal The largest Canadian 
peacekeeping operations in recent times 
— Somalia and the former Yugoslavia — 
have involved continuing conflict between 
the warring parties and violent acts 
directed at Canadian and other 
peacekeepers. For Bosnia, Canada made 


———— 
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contingency plans to remove its United 
Nations forces under fire. 


7.8 In January 1996, there were 
2,040 Canadian Forces members actively 
deployed on 12 missions including the 
NATO Implementation Force (IFOR). 
Peacekeeping has become an important 
operational concern of the Canadian 


. Forces, as evidenced by the government’s 


increasing willingness to commit naval 
and air assets. The Department is 
requesting funds for improved equipment 
for the Land Force. Canada’s policy is to 
join with other countries in pursuing 
global peace by improving global 
relations and arms control, and resolving 
regional conflict. 


7.9 In the 1994 Defence White Paper, 
the government gave the Department the 
task of providing effective, realistic and 
affordable defence services to meet 
national objectives and international 
commitments. The White Paper states, 
“Canada needs armed forces that are able 
to fight ‘alongside the best, against the 


Peacekeeping casualty (see 
= ‘| paragraph 7.14). 
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best.” It must do this despite budget cuts 
and in a fluid, multipolar world. 


7.10 The Department has tasked the 
Maritime, Land Force and Air Command 
with generating the forces promised to the 
United Nations for peacekeeping 
missions. This means providing the 
necessary personnel, equipment and 
training. Normally, no additional funding 
is provided to the Department for 
peacekeeping operations. 


7.11 
the Department of National Defence must 
train and equip its forces to an adequate 
level, assess requests from the United 
Nations and allies for assistance and 
recommend appropriate military 
involvement. It must deploy and sustain 
peacekeeping forces while ensuring that 
other defence tasks can also be met. The 
requirements created by Canada’s 
involvement in peacekeeping therefore 
involve a significant portion of the 
Department’s activities in Canada. 
Peacekeeping is only one of the many 


To participate in peacekeeping, 


activities that use the Canadian Forces. 
Our audit therefore included various 
issues involved in generating multipurpose 
forces, such as the adequacy of 
equipment, training and medical support. 
The overall objective of our audit was to 
assess the quality of management 
practices related to peacekeeping rather 
than to assess military judgments or 
achievements. Further details on our audit 
scope, objectives and criteria are included 
at the end of the chapter in the section 
“About the Audit”. 


Observations and 
Recommendations 


Peacekeeping Achievements 


Canada has made significant 
contributions 


7.12 The Canadian Forces have taken 
a leading role in United Nations 
peacekeeping efforts for many years. To 
gain an appreciation of the contribution of 
the Canadian Forces we selected 


Canadian Forces member 
assists wounded Rwandan 
child (see paragraph 7.18). 
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peacekeeping missions to Somalia, Haiti, 
Angola, Rwanda and the former 
Yugoslavia. 


Talo February 1993 saw the peak 
Canadian presence and participation in 
peacekeeping operations in those 
missions, representing six percent of the 
Canadian Forces. 


7.14 As of January 1996, there were 
approximately 863 Canadian military 
personnel deployed in active United 
Nations peacekeeping operations, 
representing three percent of all military 
personnel under United Nations 
command. There were also 1,000 
personnel committed to NATO. Since 
1990, the Canadian Forces have suffered 
116 casualties in peacekeeping operations, 
including 16 killed. 


7.15 Mission tasks. The Department 
of National Defence conducted an overall 
evaluation of peacekeeping operations up 
to 1990, but did not evaluate individual 
missions. Although there still is no formal 
evaluation of the Canadian contingents’ 
achievement of their tasks in United 
Nations or combined operations, we found 
considerable evidence of their 
accomplishments. 


7.16 In Somalia, the mission objective 
was to establish a secure environment for 
the conduct of humanitarian relief 
operations. The Canadian Forces 
contributed humanitarian efforts by 
providing medical personnel to assist local 
hospitals in Mogadishu and Belet Uen 
before the arrival of a tactical team; by 
gaining control of the airport in Belet Uen 
and re-establishing the local police there; 
by repairing abandoned equipment and 
building and reconstructing roads for 
transport vehicles in Belet Uen; and by 
educating local civilian authorities in 
mine detection and avoidance. Individual 
Canadian Forces members also used their 
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rest time to participate in other civil 
development projects. 


TAT In Haiti, the mission objective 
was to help the democratic government 
maintain a stable and secure environment. 
The Canadian contingent was instrumental 
in helping to organize free and fair 
elections without disorder. 


The Canadian Forces 
7.18 In Rwanda, the first task of the 


Canadian Forces was to reopen the airport have achieved 

within 48 hours of their arrival. Then they significant success in 
opened a medical clinic in the town of : . 
Kibungo to help the United Nations High {Del peoeneuldere 
Commissioner for Refugees and on peacekeeping 
non-governmental organizations treat the missions. Missions 
local population. They also assessed, 
planned and repaired a number of have been 

buildings to be used by other contingents distinguished by many 
oo humanitarian agencies. On HG own individual acts of 
initiative, the Canadian troops actively ; 

supported over 6,000 children in generosity and 


orphanages. bravery. 


7.19 Individual achievements. 
Canadian involvement in peacekeeping 
missions has earned our country a strong 
international reputation for excellence. A 
significant reason for its success has been 
the leadership, professionalism and 
bravery of Canadian soldiers. These traits 
are exemplified in the numerous awards 
given by the Governor General to 
Canadian soldiers for their meritorious 
service and bravery. 


7.20 In our sample of missions, over 
114 soldiers and officials were decorated 
for their military deeds, their outstanding 
professionalism, their devotion to duty or 
their acts of bravery during peacekeeping 
service. Exhibit 7.1 provides two 
examples. 


7.21 Our numerous interviews with 
United Nations staff revealed a clear 
consensus on the high value of the 
Canadian Forces’ contributions to United 
Nations peacekeeping operations. 
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Exhibit 7.1 


Examples of Personal Bravery 


Military Participation and 
Information for Decision Making 


There have been gaps in deployment 
planning 


7.22 One audit objective was to assess 
whether staff assessments were adequate 
to provide military advice on whether and 
how Canada should participate in 
peacekeeping missions proposed by the 
United Nations. We reviewed files on our 
sample of missions to assess the process 
that leads to a decision to contribute 
troops. 


7.23 National Defence procedure for 
supporting such decisions varied 
considerably from mission to mission, and 
with each rotation. The Department 
co-operates closely with staff of Foreign 
Affairs and International Trade and the 
United Nations, in a flexible process that 
depends on the size and timing of the 
proposed mission. The Minister of 
National Defence or Cabinet makes the 
final decision to contribute troops when a 
formal request is made by the United 
Nations. 


7.24 Before issuing a formal request, 
the United Nations makes an informal 
request to Foreign Affairs through its 
permanent mission to the UN. The 
Department forwards the request to 
National Defence, which conducts a “staff 
check” on its ability to contribute within 
the United Nations parameters for the 


mission, and on any other considerations 
important to the Canadian government. 


Tide We expected that advice to the 
Minister or Cabinet on a proposed 
deployment of Canadian troops would be 
based on a clear military estimate of the 
likelihood that the peacekeeping mission 
overall would succeed. We expected to 
find a clear assessment of the likelihood 
that the tasks proposed for Canadian 
troops were feasible and that the 
Department would assess impacts on 
readiness to meet other commitments. 
(These concepts are found in every recent 
version of defence policy guidelines for 
participation in peacekeeping.) We also 
expected to find that the Canadian Forces 
would assess their success in performing 
the assigned tasks and use that 
information in making decisions on 
subsequent rotations to the same mission. 


7.26 While consensus on the size and 
mandate of a possible mission is being 
formed at the United Nations, the 
departments of Foreign Affairs and 
National Defence may be called on to 
consider a variety of scenarios in a short 
period. The Security Council may choose 
an option significantly different from 
those considered. The UN then issues to 
the Canadian government a formal request 
to participate. The Canadian government 
makes its final decision based on the 
formal request. A reconnaissance mission 
to the theatre of operations is then 
conducted to gather information on 
conditions and logistical requirements. 


A pair of Canadians demonstrated professionalism and bravery by piloting a Sea King 
helicopter to evacuate a critically wounded Somali and a surgical team. An ongoing battle was 
raging in the city. Because of the danger, other forces refused to undertake the mission, but the — 


Canadians elected to attempt the evacuation. 


Another Canadian rescued three children ae were under direct fire near Visoko, 
Bosnia—Herzegovina. From the observation post where he was on duty, he noticed the children 
hiding behind the gates in a cemetery. He immediately called for a driver to move an armoured 
personnel carrier down the road while he ran along the side, protected from the direction of the 
firing. He then broke cover and hurried the children safely up the ramp into the vehicle. 
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PARLIAMENTARIANS WANT BETTER INFORMATION ON PEACEKEEPING OPERATIONS 


he importance and scope of 

peacekeeping over the last few years 
has generated considerable interest among 
parliamentarians. The current government 
has made a practice of holding a 
parliamentary debate before sending 
sizable contingents overseas on — 
peacekeeping missions. 


Chapter 6 of our 1992 Report set out three 
criteria for government departments 
providing information to Parliament: 
relevancy, reliability and 
understandability. That report also stated 
that information for Parliament should 
answer four questions: 


@ What are the department’s mission 
and lines of business? 

e@ How does the department carry 
out its lines of business to achieve its 
mission? . 

@ What are the department’s 
strategic objectives for realizing its 
mission, and plans for managing the 
significant public resources under its 
control? 

e@ How did the department do in 
meeting its objectives, and how much did 
it cost? - 

In this audit, we conducted a review of 
two separate years of documentary 
information provided to Parliament: the 
fiscal years ended 31 March 1993 and 31 
March 1995. Our document review 
focussed on information provided about 


| 
| 


criteria for participation in a mission, 
equipment to be deployed or being used, 
accomplishment of tasks, training and 
availability of personnel. 


Our review covered a wide variety of 
documents: the House of Commons 


_ Debates, testimony before Senate and 


House committees, Senate Debates, Part 
III of the Estimates, the 1992 Defence 
policy statements and the 1994 Defence 


_ White Paper. The table below summarizes 


our findings. 


The contradictions we noted were related 
to equipment issues, criteria for 
deployment and accomplishment of tasks. 


We also conducted interviews with 
parliamentarians from all major parties 


| who sit on the Standing Committee for 
| National Defence and Veterans Affairs. We 


asked them whether they felt our four 
questions were being properly addressed 
in information provided by Foreign Affairs 
and National Defence. We also asked them 
whether the departments provided 
information soon enough. 


Their comments paralleled our document 


__| analysis. Members felt that what 
| information they did receive was generally 


relevant, reliable and understandable, 
taking into account that it was very often 
incomplete. Some noted that the lack of 
security clearance for MPs was a definite 
barrier to obtaining complete information. 


MPs also noted that the departments 
tended to provide information on a topic 
only when specifically asked. Most 
observed that departments did not provide 
the information quickly, especially for the 
House debates on troop contributions. 
They were unanimous in asking for better 
information on the actual costs of each 
mission. They also said they were not 
provided with information on 
departmental strategic planning for 
peacekeeping or with information on 
evaluation of task achievements. 


Officials noted that the government had 
greatly increased briefings to 
parliamentarians and the opportunity to 
debate United Nations commitments. They 
also noted that they respond to all 
individual inquiries received from 
individual members. 


Government should consider whether the 
role contemplated by Parliament is such 
that there should be a mechanism that 
would allow MPs concerned with troop 
deployments greater access to classified 
documents while respecting parliamentary 
privilege. The Department should work 
with the Standing Committee on National 
Defence and Veterans Affairs to establish 
an information framework that will 
address the questions and criteria 
enunciated in Chapter 6 of the Auditor 
General’s 1992 Report. This framework 
should enable MPs to obtain quickly all 
required information. 


rc 


Summary of Information Provided to Parliament 


Information to Parliament 
on Peacekeeping 


Relevant Reliable 


Understandable 


Mission and activities of 
the department 


How the department carries out 
its lines of business 


Strategic objectives and plans 
for managing resources 


| Achievement of objectives 
and cost 


Yes, but incomplete 


No substantive 
information provided 


No substantive 
information provided 


No substantive 


No substantive 


Contradictions noted 


information provided 


information provided 


No substantive 
information provided 


No substantive 
information provided 
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cho) Departmental officials told us 
that headquarters staff meet to appraise 
whether there is a clear and achievable 
mandate; whether the mission and 
Canadian tasks are likely to succeed; the 
potential risks to Canadian peacekeepers; 
and the impacts of a contribution on 
defence policy commitments, training and 
readiness. They also told us that their 
assessments are often not written, in large 
part because discussions of the Canadian 
role and extent of contribution must be 
held informally with United Nations staff 
before the actual mandate is approved by 
the Security Council and before the troop 
tasks have been identified. The short 
timelines involved also mean that much 
co-ordination takes place in face-to-face 
meetings or by teleconferencing. Officials 
also told us that they usually conduct such 
assessments for the first deployment. 
Ongoing field reporting and other 
information is used for later rotations. 


7.28 Planning for deployments on 
United Nations peacekeeping missions can 
be very fluid, often within a compressed 
time period. In addition, planning is 
complicated by United Nations manning 
ceilings that require planners to modify 
Canadian military units. In our sample of 
missions, we found that National Defence 
did not conduct a comprehensive written 
assessment for each rotation on the 
feasibility and likelihood of success of the 
overall mission. We did find, however, 
that during the course of our audit the 
Department issued a staff instruction, and 
anew draft staff procedure requiring this 
is now being followed. 


7.29 Our examination of mission files 
found evidence of gaps in planning for 
deployments. For example: 


¢ For the initial deployment to the 
Balkans in the spring of 1992, we could 
find no evaluation of the likelihood that 
the overall mission would succeed, 


although the probable Canadian tasks 
were studied. A proper check of the 
Forces’ capability to contribute troops was 
made, but due to time constraints a full 
reconnaissance was not carried out. 
Although a reconnaissance mission was 
conducted for the second rotation to 
Bosnia, the departmental post-operations 
report criticized its methodology. A full 
staff study of the situation in the Balkans 
and Canada’s potential future 
commitments was conducted in August 
1992. Another study, which assessed the 
mission’s progress, was performed in May 
1993. Beyond these studies, the 
Department did not analyze each rotation 
systematically. 


e There was no full reconnaissance 
mission for Canada’s first rotation to 
Rwanda, despite the fact that Canadian 
troops were taking on entirely new tasks. 
A full reconnaissance mission was 
conducted for the second rotation, but 
subsequent situation reports show that 
many of the problems it had raised about 
the tasks assigned to Canada were not 
addressed. 


e Canadian contingents provide 
operational information to National 
Defence Headquarters through situation 
reports and post-operation reports. These 
reports, however, seldom comment on the 
overall success of the mission. Although 
valuable information on the success of 
Canadian efforts was often sent to 
headquarters in the missions we 
examined, it was not then a requirement 
and it was not always evident how the 
information was used in planning future 
rotations. 


7.30 All military staff are expected, as 
they conduct staff checks, to consider the 
departmental guidelines on participation 
in peace support operations for 
deployment. After issuing its staff 
instruction in 1995, the Department used a 
more systematic approach in planning the 
most recent deployment to Rwanda. 
Officials told us that reports received from 
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the field are seen and reviewed all the way 
to the level of the Chief of the Defence 
Staff. 


IPAM With regard to the military advice 
that was given to the Minister, our access 
to Cabinet documents, as provided for in 
the 1985 Order-in-Council, did not enable 
us to undertake a complete analysis. 
Sufficient documentation regarding 
UNPROFOR was obtained to determine 
that ministers had received some 
information relating to the likelihood of 
success of the overall mission, the 
feasibility of the Canadian role, the 
impact on the Canadian Forces, and the 
risk to Canadian personnel. 


fp 9 The Department should 
continue with and finalize its 
improvements to staff procedures for 
mission planning. 


Department’s Response: The Department 
concurs with the recommendation. Action 
is ongoing and particular attention is 
focussed on actioning lessons learned in a 
more timely fashion so that they can be 
used to support new missions and 
subsequent rotations to existing missions. 


Force Generation 


Vs35 The level of peacekeeping 
operations the Land Force has conducted 
since its deployment to the former 
Yugoslavia has tasked it to the limit of its 
designed ready capability. The 1994 
Defence White Paper set the goal of being 
able to deploy a force of 4,000 naval, land 
and air force troops on three weeks’ notice 
and then sustain it indefinitely in the field 
in a low-intensity environment. At the 
peak of its peacekeeping duty in 1993, the 
Land Force was at, or above, this White 
Paper goal. 


7.34 It is current government policy to 
maintain a multipurpose combat force. 
The 1994 White Paper contains the stated 
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goal that such a force would be able to 
operate with the modern forces 
maintained by our allies and be able to 
fight “alongside the best, against the 
best.” 


Le3S The Report of the Special Joint 
Committee on Canada’s Defence Policy, 
which preceded the 1994 White Paper, 
stated: 


Heavy operational demands on 
the Land Forces, coupled with 
the fact that units have been 
manned at only one-half to 
two-thirds of their established 
strength, have meant that combat 
units have been brought up to 
strength with reservists or 
soldiers from other units. Some 
individuals have had two or more 
tours in two and one-half years. 
The cost in lowered morale, early 
departures and reduced efficiency 
is considerable. 


7.36 The report went on to 
recommend: “The solution to the problem 
is to increase the manning level of land 
force combat units in Canada.... This will 
require 2,500 ... combat personnel ... a 
further 1,000 ... to upgrade engineers, 
signals and other combat support units.” It 
estimated the cost at an additional 

$210 million per year, with capital 
investments of $700 million over five 
years. The 1994 White Paper translated 
this recommendation into an additional 
3,000 soldiers for the army’s field force. 


TEM We set out to ascertain whether 
the Canadian Forces could maintain its 
other readiness commitments under the 
1994 Defence Policy while generating the 
ready forces for peacekeeping. We wanted 
to ascertain whether certain elements of 
personnel were being overemployed and 
whether equipment was in danger of 
becoming unavailable. 
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Exhibit 7.2 


7.38 We focussed on the key areas of 
personnel, use of the Reserves, medical 
support, and equipment. To what extent 
are Land Force personnel being 
overtasked by peacekeeping? Is effective 
use being made of the Reserves? Does the 
medical support restructuring now being 
planned support the White Paper 
commitments? Is there enough equipment 
to support peacekeeping operations and 
still maintain readiness for other 
commitments? Is the equipment supplied 
to our peacekeepers adequate? 


Tasking of personnel 


7.39 We expected the Canadian Forces 
to have the trained personnel needed to 
allow them to reasonably sustain current 
peacetime obligations according to stated 
defence policy, and to allow them to be 
ready to respond adequately to 
contingencies, including peacekeeping 
missions. 


Number of Peacekeepers 


7.40 We summarized personnel data 
from the Military Personnel Information 
System on peacekeepers who served 
between 1990 and early 1995. We 
compared this with the trained effective 
strength of the Canadian Forces. During 
the analysis we noted that the information 
system does not track the peacekeeping 
service of air crews or others on 
temporary peacekeeping duty. Since we 
were not able to identify a ready source of 
these data, our conclusions do not extend 
to these groups. To reach our conclusions, 
we conducted an exhaustive analysis of 
each occupation and rank involved in 
peacekeeping during the five-year study 
period. We did not examine the frequency 
of deployment of army units; we focussed 
on how the personnel are used. 


7.41 Overall services. Only a small 
portion of the Regular Force were serving 
as peacekeepers at any one time. Until 
1992, peacekeeping service was limited to 
less than 1,000 members of the Force, 
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roughly one percent. Following the 
closing of bases in Germany, service 
peaked in early 1993 at almost 

4,000 members, or almost six percent of 
the effective strength. Since 1993, service 
has decreased and at the end of 1995 was 
around three percent, or 2,400 members. 


7.42 Regular Force and Reserve 
Service. Exhibit 7.2 shows the peak of 
service by the Regular Force in early 
1993, followed by a peak of service by 
Reservists later that year. At least one 
peacekeeping unit was composed of more 
than 40 percent Reserves during that year. 


7.43 Service by occupation and 
rank. There have been many anecdotal 
statements about certain units or 


Occupation 


Cook 
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individuals in the Canadian Forces who 
were sent for three or four tours of duty. 
The Land Force chain of command was 
deeply concerned that should the situation 
in the former Yugoslavia persist, a 
continuous cycle of tours would become 
the norm and would have severe personal 
consequences for its members. However, 
our analysis indicates that this has not 
occurred. 


7.44 Exhibit 7.3 shows the most 
severely tasked non-commissioned 
occupations, showing the average number 
of days served between 1990 and 1995 for 
each occupation. 


7.45 A typical non-commissioned 
member of the Regular Force has done 
just over 50 days of peacekeeping duty 


Exhibit 7.3 


Average Number of Days 


Fire Control Systems 
Technician (Electronic) 


Refrigeration and Mechanical 
Technician 


Preventive Medicine 
Technician 


Spent on Peacekeeping 
between 1990 and 1995 


Even the most severely tasked 
occupations were not 
overburdened. 


Radio Operator 


Structures Technician 


Postal Clerk 


Electrical Generating 
Systems Technician 


Electrician 


Armour Crewman 


Field Engineer 


Plumber Gas Fitter 


Operating Room Assistant 


Infantryman 


Field Engineering 
Equipment Operator 


Number of Days over a Five-Year Period 
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Although there are 
exceptions, we found 
that, overall, 
peacekeeping has not 
put an excessive 
burden on individual 
members of the 
Canadian Forces. 


during the five years studied. Some of the 
almost 100 non-commissioned 
occupations provided only one or two 
days per member. In contrast, field 
engineering equipment operators served 
an average of 200 days per member. Eight 
of the fifteen most heavily tasked 
occupations were engineering 
occupations. Other occupations more 
frequently called on were those of 
infantryman and crewman. 


7.46 We found that officers (excluding 
air crews, for whom data were not 
available) provided slightly less service, 
with an average of 40 days per member, 
than did other ranks. The combat arms 
occupations of artillery, armour and 
infantry officer were the most heavily 
tasked. Armour officers provided the most 
service, an average of 130 days each. 


7.47 For non-commissioned members, 
the level of service decreased among 
higher ranks. The most active ranks were 
corporals and privates, with an average of 
slightly more than 55 days service per 


member. The least active rank was chief 
warrant officer, with only 15 days per 
member during the period considered. 


7.48 In contrast, officers displayed a 
more even level of service. Except for 
lieutenants, officers typically served about 
40 days over the five years studied. 


7.49 Frequency and length of 
service. In addition to analyzing whether 
the extent of service is evenly distributed 
over a particular rank or occupation, we 
also examined taskings by individual. 


7.50 We found that officers and other 
ranks displayed similar distributions of 
peacekeeping service. The majority of 
peacekeepers had performed between 150 
and 200 days of service, indicating one 
tour lasting six months (see Exhibit 7.4). 


Heal Similarly, officers and other ranks 
displayed the same distribution of number 
of tours (Exhibit 7.5). Roughly 80 percent 
of personnel had served only one tour 
during the period studied. However, more 
than 3,000 personnel had two or even 
three tours during the period studied. 


Exhibit 7.4 Number of Peacekeepers 
Distribution of Number of 12.000 ~~ 
Peacekeeping Days per 
Non-Commissioned 
Peacekeeper 1990-1995 10,000 — 
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Most members of the forces 
performing peacekeeping duty 
served about six months. 6.000 — 
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There were no data on how many of this 
group had volunteered for additional 
tours. 


1.52 We evaluated the 200 individuals 
who had the most peacekeeping service in 
number of tours and length of duty. These 
peacekeepers came from the occupations 
indicated in Exhibit 7.6. The exhibit 
clearly shows that some individuals, in 
particular those from the Army’s combat 
arms and combat service support 
occupations, are providing significant 
peacekeeping service, far beyond the 
norm for their occupations. 


7.53 Impact of Force downsizing. 
During the period studied, the size of the 
Regular Force went from roughly 85,000 
to 70,000 and is expected to eventually 
reach 60,000. To determine the potential 
impact of downsizing on peacekeeping, 
we compared the numbers of trained 
personnel estimated for 31 March 1996 
with the numbers at March 1993 (see 
Exhibit 7.7). 


7.54 We noted similar results for both 
officers and other ranks. No occupation 


Number of Peacekeepers 
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showed what might have been an 
excessive burden. While the Canadian 
Forces have managed the burden of 
peacekeeping operations on individuals, 
problems still exist in attaining collective 
training goals (see paragraphs 7.96 to 
7.111). Overall, we found that the 
Canadian Forces did not place undue 
demands on any one group to serve on 
peacekeeping tours. However, as early as 
1993, the Land Force become concerned 
that sustaining two battle groups in the 
Balkans would make the maintenance of 
unit cohesion difficult on rotation and 
would overtask individuals. 


755 The 1994 White Paper allowed 
an additional 3,000 troops for the Land 
field force because of renewed emphasis 
on multilateral operations. These troops, 
now estimated to cost about $135 million 
a year, were to be reallocated from cuts to 
overhead occupations. The Department is 
achieving its goal by adding the 3,000 
troops to its planned 1997 field force size. 
As the existing field force was already 
1,600 over the 1997 projections, this 
meant that it would have to transfer only 


Number of Tours 


Exhibit 7.5 


Distribution of Number of 
Peacekeeping Tours per 
Non-Commissioned 
Peacekeeper 1990-1995 


About 80 percent of 
peacekeepers performed one 
tour of duty. 


Source: DND 
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Exhibit 7.6 


1,400 troops from existing overhead. We 
compared the strength of the field force in 
November 1994, when the White Paper 
was prepared, with the strength at 
December 1995. We found that over that 
period the field force had increased by 
only about 50 members. 


7.56 The Land Force grew from an 
average strength of 19,501 in 1992-93, 
when peacekeeping commitments were at 
their peak, to 21,797 in 1994-95 and was 
planned to grow to 23,862 in 1995-96. 
The Department was unable to provide us 
with a comprehensive plan for 
implementing the enhancements to its 
field force. However, according to 
officials, the Department will attain its 
targets on schedule by 31 March 1997. 


Occupation 


Maritime Engineering Mechanic 


fro! National Defence should clarify 
its objectives and plans for providing 
additional Regular Land Forces in 
order to achieve the most economical 
solution to its operational requirements. 


Department’s response: The operational 
enhancement of the field force is designed 
to raise the manning level of units. One of 
the effects will be to reduce what had been 
the prevalent practice of augmenting units 
selected for peacekeeping duty with sub 
and sub-sub units drawn from other 
Regular Force units (without eliminating 
the need for augmentation by the 
Reserves). This will reduce the disruption 
to collective training and to other 
activities aimed at maintaining the 
readiness of the Land Force. Ongoing 
planning for restructuring the Reserves, 
recommended by the Special Commission 


Average Number of Peacekeeping 
Days by Regular Force 
Non-Commissioned Members with 
Three or More Tours 


Maritime Engineering Technician 
Maritime Engineering Artificer 


Most individuals serving three or Marine Electrical Technician 


more tours come from a small 


: Bosun 
number of occupations. 


Supply Technician 
Steward 

Cook 

Medical Assistant 
Military Police 
Administrative Clerk 
Radio Operator 
Field Engineer 

Field Engineering Equipment Operator 
Armour Crewman 
Artilleryman 


Vehicle Technician 


Source: DND 
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on Restructure of the Reserves, will 
address the issue of their operational 
capability. 


Use of the Reserves 


7.58 For the foreseeable future, the 
Militia will be useful only in providing 
individuals of lower rank to augment 
peacekeeping service. For the period 
reviewed, from 1990 to 1995, the Land 
Force employed a peak of 600 Reservists 
at one time on peacekeeping duty. On 
average, it used about one percent of the 


strength of the Reserves for peacekeeping. 


The Report of the Special Commission on 
the Restructuring of the Reserves showed 
total effective strength of the Reserves at 
over 26,400 in 1995. The Militia 
accounted for approximately 18,300 of 


Occupation 


Field Engineering 
Equipment Operator 


Field Engineer 
Infantryman 
Electrician 


Signals Lineman 


Operating Room Assistant 


Fire Control Systems 
Technician (Electronic) 


Artilleryman 


Structures Technician 


Electrical Generating 
Systems Technician 


Plumber Gas Fitter 


Armour Crewman 
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this total. The average number of Militia 
members serving on peacekeeping duty 
since January 1993 has been just under 
300. The Militia has represented more 
than 90 percent of Reservists serving at 
any one time. 


Ps We set out to determine whether 
Militia members were being used in the 
most cost-effective way. We contacted 
senior officials at all Land Force area and 
brigade headquarters and consolidated 
their views on the proper use of Reserves. 


7.60 We found that the many systemic 
problems in the Reserves that we noted in 
Chapter 18 of our 1992 Report still 
prevent the Militia from contributing 
beyond augmenting with individuals at the 
lower ranks. In its current state, the Militia 


Exhibit 7.7 


Average Peacekeeping Days per 
Strength at March 1993 and per 
Estimated Trained Personnel 
Available at March 1996 


Cutbacks in strength would not 
have overburdened any particular 
military occupation. 


1996 
1993 
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D epartmental documentation shows 
that up to 20 percent of Canadian 
peacekeepers may become stress 
casualties, suffering from physical and 
mental disorders as a result of their duty. 
The nature and level of stress experienced 
can vary widely according to the 
circumstances of the individual 
peacekeeping mission. The Canadian 
Forces have studied the problem and have 
begun programs to reduce and treat stress 
casualties. The success of these programs 
will depend on the level of support given 
by commanders at all levels. 


Experts regard stress as a continuum and 
believe that different levels of stress are 
handled differently by different people. 
Typical reactions to personal stress include 
increased illness, increased drug and 
alcohol abuse, job dissatisfaction and 
depression. Viewing an horrific event can 
lead to post-traumatic stress disorder in 
which the individual’s life becomes 
wrapped up in the event. The obvious 
organizational impacts of lowered 
productivity and effectiveness are the 
outcomes of having troops who suffer from 
excessive stress. Experts agree that the 
effects of stress, particularly in military 
situations, can be lessened by good 
training, cohesive units, trustworthy 
leaders and stable family life. 


The prevention of stress casualties is 
important to military commanders, if only 
for reasons of military effectiveness. 
During World War II, the United States 
Army estimated that the ratio of stress 
casualties to physically wounded among 
American soldiers was 33 percent in 
Okinawa, 35 percent in the European 
campaign and 25 percent in North Africa. 
Estimates for Israeli soldiers in the 1973 
Arab-Israeli war were 23 percent, and 
statistics for Israeli soldiers fighting in 
Lebanon in 1984 show that fewer men died 
than were incapacitated by stress reaction. 


The Department of National Defence 
began studying combat and peacekeeping 
stress in the late 1980s. Research fell 
victim to funding cuts but, in response to 
the current levels of peacekeeping stress, 


DEALING WITH PEACEKEEPER STRESS 


the Department is now conducting new 
studies and developing new programs to 
manage the problem. 


We reviewed the stress management 
research and programs in place at the time 
of our audit. We expected that the 
Department would be able to determine the 
level and sources of stress associated with 
peacekeeping operations; that it would 
have programs and practices to deal with 
stress before, during and following 
assignments; and that it would determine 
the effectiveness of its programs and 
practices over time. 


Factors in stress 


Leadership. Departmental research in the 
area of peacekeeper and Canadian Forces 
morale has defined several key issues 
related to operational stress. First and 
foremost of these is leadership. 
Subordinates’ perception that a leader can 
protect the interests of the group and its 
members influences the amount of trust 
and co-operation given to the leader. One 
study on peacekeepers confirmed that 
good leadership is important for managing 
both organizational and individual stress. 
Difficulties with leadership were found to 
closely correlate with the physical 
symptoms of stress reported by the soldiers 
the Department studied. One major 
difficulty identified during Exercise 
VENOM STRIKE (see paragraph 7.97) 
was the belief of many soldiers and leaders 
that if they applied the VENOM Rules of 
Engagement and used deadly force when 
the Rules of Engagement permitted it, they 
would not necessarily be supported by 
their commanders if their action was then 
reviewed. 


Safety concerns. The second most 
important finding of departmental studies 
is concern by peacekeepers over their own 
and their comrades’ safety. General 
concerns, or repeated incidents, can raise 
the overall stress levels for peacekeepers 
and result in health problems. In addition, 
specific life-and-death incidents can in 
themselves directly create post-traumatic 
stress disorder, the most severe and 
disabling variation of occupational stress 


known. One departmental study reported 
on post-traumatic stress disorder in a 
mixed group of Regular and Reserve Force 
peacekeepers six months after their 
mission. The study found that “probable” 
post-traumatic stress disorder rates for the 
group were between 10 and 20 percent. 


Media coverage. A third finding of the 
departmental studies concerns the impact 
of the media. Certain issues, such as the 
way the media have presented the 
Canadian Forces in cases such as the 
Somalia mission, the hostage taking of 
Canadian peacekeepers, and the 
disciplinary problems of the Canadian 
Airborne Regiment have been a source of 
significant dissatisfaction and stress to 
members of the Canadian Forces. The 
impact of media coverage perceived as 
unbalanced is two-fold: it causes Canadian 
Forces members to feel misunderstood or 
unappreciated, and it is seen as having an 
unsettling effect on family members. The 
studies indicate that media stories are 
partly responsible for stress-related health 
problems found in peacekeepers and the 
perceived levels of anxiety among family 
members. 


The Department’s actions 


The Department is taking action in 
response to its research findings. It has 
begun, or is continuing, a number of 
activities to aid in the management of 
stress. These include: 


@ understanding and measuring the 
sources of peacekeeping stress; 


@ screening members for their 
military readiness before sending them on 
a mission; 

@ continuing military training that 
can effectively inoculate soldiers against 
peacekeeping stress. High levels of unit 
morale, stamina, technical competence, 
familiarity with and confidence in weapon 
systems, and field training are all known to 
reduce stress in soldiers; 


@ increasing the manning of the 
Army field force, as required by the White 
Paper; 
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e limiting the participation of 
Reservists in peacekeeping units to 20-25 
percent of unit strength to increase unit 
cohesiveness; 

@ initiating predeployment stress 
training, in-theatre support programs and 
repatriation debriefings for deployed units; 
and 

© continuing research into stress, 
including the development of videos and 
brochures on stress. 


The challenge for commanders 


Department to put programs in place to 
manage stress, departmental studies 
indicate that these programs will not be 
fully effective unless commanders at all 
levels demonstrate increased support for 
behaviours that reduce and help manage 
stress. Departmental studies show a major 
impediment to the management of stress is 
an unwillingness to discuss issues 
surrounding fear of injury or death. Many 
personnel suffering from post-traumatic 
stress disorder believe themselves to be in 
good emotional health and do not seek 
assistance for their symptoms. 


Departmental health care officials told us 
that clear and active leadership by senior 
officers is required to reverse these 
attitudes and implement the many concrete 
recommendations made in the studies. 


We believe the Department should 
continue to develop and implement a 
program to manage stress among its 
peacekeepers and seek to implement the 
recommendations of its research programs. 
The active involvement of senior 
management and command personnel will 
be necessary. 


In spite of the serious efforts by the 


cannot support a collective or unit 
augmentation. Turnover, the lack of time 
to train, and low turnout for training due 
to school or job pressures all contribute to 
a very low level of collective and 
leadership training. In addition, the 
problem of voluntary turnout for a given 
mission effectively impedes the formation 
of cohesive units. 


7.61 The use of Reservists places 
particular demands on force generation for 
peacekeeping operations. Reservists must 
be trained and evaluated separately prior 
to predeployment training. Support-trade 
Reservists sometimes require extra 
supervision in the field. Post-operation 
reports commented on the utility, quality 
and training of Militia. Key observations 
include: the “enormous effort” required at 
all levels of command to generate and 
train the required number of Militia 
soldiers, the lack of leadership skills 
among the Militia and the requirement for 
additional supervision in theatre for some 
ranks and trades. On Land Force 
Command troop rotations trained in 1994 
and 1995, about 20 percent of Militia 
personnel selected for peacekeeping 
missions were unable to pass the selection 
training and evaluation at the lowest level 
of required individual skills. As a result, 


targets for participation of Militia soldiers 
in contingents deployed are often not met. 
Contingents routinely deploy with fewer 
than the authorized number of Reservists 
because, once training attrition is 
accounted for, too few Militia members 
volunteer and pass personnel screening to 
fill all the slots available. 


7.62 The recommendations made in 
Chapter 18 of the Auditor General’s 
1992 Report are still essentially valid. 
The Department of National Defence 
should ensure that the many issues 
relating to the Reserves, and especially 
the Militia, are resolved quickly. 


Department’s response: A number of 
initiatives in the areas of recruiting 
procedures, personnel policies as well as 
pay and benefits address the main 
recommendations of the Auditor General’s 
1992 Report. Additional recommendations 
flowing from the Special Commission on 
the Restructure of the Reserves, the 
related reports from the Standing 
Committee on National Defence and 
Veterans Affairs and the Senate Standing 
Committee on Social Affairs, Science and 
Technology are scheduled for 
implementation later this year. 


Medical support to operations 


7.63 The Department of National 
Defence is prepared to accept the risk of 


About 20 percent of 
Militia personnel 
selected for 
peacekeeping 
missions were unable 
to pass selection 
training and evaluation 
at the lowest level of 
required individual 
Skills. 
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not being able to meet medical support 
goals for Land Force operations. 
Chapter 23 of our 1990 Report noted that 
although the military hospital system in 
Canada was severely underused, the 
medical support system could not meet 
wartime requirements. A departmental 
program evaluation confirmed the 
peacetime focus of the Canadian Forces 
medical system. Cutbacks to departmental 
funding made reform imperative. 


7.64 The Department is currently 
completing plans that aim at an 
operationally oriented, viable and 
cost-effective Canadian Forces medical 
support system. The focus has thus shifted 
to operational capability from peacetime 
care. We expected the new system to be 
capable of supporting assigned Canadian 
Forces missions and tasks, including 
peacekeeping. While most peacekeeping 
assignments result in few Canadian 
casualties, there can be a danger that 
missions could escalate into more serious 
conflict involving an intervention force. 


7.65 Our audit uncovered no instances 
to date of inadequate medical support for 
peacekeeping operations. However, the 
nature of operations in support of peace is 
evolving. There are more uncertainties 
and risks of possible escalation or 
unforeseen violence that would require 
medical support. The Department has no 
specific plans for medical support to 
peacekeeping, although it prepares a 
medical support plan for each individual 
mission. Its plan addresses force 
generation for multipurpose combat goals, 
as stated in the White Paper. We therefore 
concentrated our analysis of the 
Department’s medical support plan on its 
ability to meet those multipurpose goals. 
We concentrated on the medical capability 
needed at “third line”: the provision of a 
field hospital where surgery would be 
performed. We reviewed published 


operational plans of the Department and 
requirements for minimum medical 
support. We also conducted interviews 
with departmental medical and 
force-structuring officials. We compared 
documented casualty-handling 
requirements with the planned capability 
of Canadian Forces medical support. 


7.66 The 1994 White Paper commits 

the Canadian Forces to be able to deploy 

anywhere in the world one or more of the 
following elements: 


e anaval task group; 


e abrigade group or three separate 
battle groups with appropriate support; 


e awing of fighter aircraft; and 


e one squadron of tactical transport 
aircraft. 


The policy also commits the Canadian 
Forces to be able to provide single 
elements or the vanguard components of 
this force within three weeks and be able 
to sustain them indefinitely in a low-threat 
environment. They must also provide the 
remaining elements of the force within 
three months. 


7.67 Third-line medical care should be 
able to provide simultaneously for these 
forces. Any assessment of options for a 
military response, for peacekeeping or 
otherwise, must include an analysis of 
likely casualty rates because of their 
operational and political significance. 
Departmental force planners have 
assumed a mid-intensity conflict for these 
commitments. Medical planners were 
instructed to be able to provide for the 
vanguard force, but to limit their planning 
to 200 beds. The current capacity is 

140 beds. Medical and force-structuring 
staffs agree that a 50-bed support hospital 
is sufficient for the operational naval 
forces and the air forces. The 
departmental medical support plan will 
meet these requirements. 
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7.68 Departmental planners relied on 
NATO best estimates for casualty rates in 
a mid-intensity conflict, and calculated 
that a third-line hospital of 340 beds 
would be required for the Land Force 
alone. This number is arrived at using a 
risk factor acceptable to NATO of 

25 percent. That is, there is a one in four 
chance that casualties will exceed the 
ability of the medical system to handle 
them. Departmental planners did not have 
any reliable estimates of casualty rates for 
a low-threat scenario such as most 
peacekeeping missions, but officials told 
us they have considerable experience. The 
planners’ final report in June 1995 noted 
that a third-line hospital of 200 beds was 
an “imposed limitation” and states, “CF 
and DND authorities will need to accept 
the associated risk.” For example, if all 
the force components were deployed 
simultaneously and each of the air and 
naval forces were allotted 50 beds, a 
200-bed hospital would leave only 

100 beds for the army instead of the 340 
calculated as necessary. 


7.69 Departmental officials note that, 
in the past, NATO operational 
commanders have not always accepted 
NATO casualty estimates in their 
planning. Even though they would be 
below the NATO standard, they are 
willing to accept the associated risk. This 
would mean that if a peacekeeping or joint 
operation reached mid-intensity and a 
brigade group of the Canadian Forces 
sustained casualties at planned rates, they 
would have to improvise 240 beds in the 
field if no spare capacity were available 
from allies. Although the Surgeon General 
staff has proposed a new medical reserve 
force to supply additional personnel, this 
proposal has not yet been implemented. 
Officials also informed us that surgeons 
and anesthetists are already at minimum 
strength to support field operations. 
Medical staff have proved difficult to 
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recruit and retain, both in the Regular 
Force and the Reserves. 


7.70 Conclusion. We conclude that 
the Canadian Forces in effect would not 
be able to deploy the minimum force 
described in the 1994 White Paper without 
arranging with allies for this key medical 
support element. Given this fact and that 
the Canadian Forces have not calculated 
the requirement for a low-intensity 
scenario, we also conclude that the 
Department has not properly justified the 
number of beds it plans to acquire. 


five) For peacekeeping operations in 
recent years, Canadian troops have often 
relied on third-line medical support 
provided by other countries. This is a 
normal method of operation for all 
participants in United Nations missions. 


Taiz Departmental officials pointed 
out to us that the plan is a marked 
improvement over previous capability. 
They also pointed out that Canada has not 
required a 340-bed capacity since the 
Korean war. While we agree that these 
facts are correct, we also make the 
following recommendation. 


Medical support 
provisions are below 
NATO standards. If 
casualties were 
sustained at the NATO 
rate for mid-intensity 
conflict, a Canadian 
Forces brigade group 
would have to 
improvise 240 hospital 
beds. 


IRIS) The Department should clearly 
define the medical risks of its proposed 
200-bed hospital with respect to White 
Paper commitments and the most likely 
operational scenarios. 


Department’s response: In addition to 
implementing the plan to develop the 
200-bed field hospital, the Department is 
identifying and investigating the 
implications of the additional personnel 
and equipment resources needed to bring 
this facility to the 340-bed level required 
to concurrently support the contingency 
commitments listed in the White Paper. 


Adequacy of equipment 


7.74 In 1995 the Canadian Forces 
concluded that long-standing 
deficiencies in equipment would not 
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Canadian Forces armoured 
personnel carriers will be 
replaced with vehicles 
similar to this Bison light 
armoured vehicle (see 
paragraph 7.75). 


allow troops to perform assigned 
missions within an acceptable level of 
risk. To determine whether Canadian 
Forces members were deployed with 
adequate equipment, we used our selected 
sample of missions. We focussed on the 
APC and Cougar major equipment fleets. 
We expected to find that equipment 
deployed would meet the intent of 
departmental policies and directives and 
be consistent with documentation 
developed under the Defence Program 
Management System. We reviewed capital 
program documentation and reports 
produced by troops in theatre. We also 
attempted to compare equipment deployed 
by other countries for similar mandates. 


q.1> Armoured Personnel Carriers 
(APCs). The Canadian Forces have 
known since 1980 of deficiencies in 
armoured personnel carriers (APCs) used 
in peacekeeping operations. In 1995, the 
Canadian Forces determined that these 
deficiencies would not allow troops to 
perform assigned missions within an 


acceptable level of risk. In August 1995, 
the government decided to replace and 
refurbish the fleet of APCs. We did not 
audit this decision. Exhibit 7.8 shows the 
potential impact of this replacement 
program. 


7.76 In justifying the purchase and 
refurbishment, departmental documents 
stated that the Canadian Forces need 
modern equipment to face all future 
requirements, including peacekeeping. 
“Canadian troops, acting as part of an 
international force for peacekeeping, 
coalition or NATO operations, are likely 
to encounter modern, technically 
sophisticated weapons and vehicles.” 
According to departmental officials, this 
deficiency had already been identified in 
capital project documentation as early as 
1980, when a replacement project was 
approved. The project was cancelled in 
1988. A similar project had been approved 
in 1986, but was cancelled in 1989. A 
further new project was approved later in 
1989 and cancelled in 1992. 
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TAd. Some of the important 


deficiencies detailed in departmental 
documentation include the following: 


e “There are serious deficiencies in 
firepower, mobility and capacity.” 


e “Protection levels against direct fire 


weapons, shaped charge threats and mines 


have proven inadequate.” 


e “Self-defence firepower is 


ineffective against hostile light armoured 
vehicles due to inadequate weapon 
calibre, exposure of the fire, and lack of 


an effective fire control system.’ 


e “Mobility is marginal.” 


7.78 According to the documentation, 


> 


none of the family of APCs meets the 
standards of combat capability necessary 
to perform the assigned mission elements 
with an acceptable risk to troops. 


fh The Canadian contingent 


commanders to UNPROFOR (the mission 


to the former Yugoslavia) confirmed the 
APC deficiencies. Even after the APCs 


were reinforced with gun shield kits, 1992 


DND documents acknowledged: “The 
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M113 APC will still be deficient in terms 

of its overall protection. Neither the APC 

itself nor the gun shield will prevent 

penetration from any round more powerful 

than the standard NATO 7.62 (rifle) ball. Long-standing 

... the M113 APC will continue to be very ga ee 
vulnerable to anti-tank mines.” As late as deficiencies in 

1994, the Canadian contingent armoured personnel 
commander to DNEROEOR, expressed carriers exceed the 
serious concerns. He indicated that the 

main threat was from individuals armed margins of risk now 
with weapons up to 14.5 mm, and that acceptable to the 
protection from that threat was “highly 
desirable’. In 1995, 85 M113 armoured 
personnel carriers, 38 Grizzly armoured 
personnel carriers and 16 Bisons were 
fitted with additional armour as a 
temporary measure to protect personnel in 
the Balkans against small arms fire and 
mines. 


Canadian Forces. 


7.80 Cougar tank trainer. The 
armoured vehicle general purpose Cougar 
is a wheeled vehicle that was purchased in 
the late 1970s as a tank trainer for 
armoured regiments and an operational 
vehicle for low-intensity operations such 
as UN peacekeeping or internal security. 


Exhibit 7.8 


Planned Land Vehicle 
Replacement 


_ Replacement program / M113 Grizzly Bison LAV 25 (8X8) Total 
APC type tracked 
Current fleet 1,214 269 199 1,682 
No refurbishment 441 6 0 447 
To refurbish WS 263 199 15235 
New acquisition 240 ordered 
120 option 1* 
120 option 2* 
171 option 3* 
651 total 240 
Delivery dates Not Not Not January 1998 to 
decided decided decided March 2000 


* Not fixed, as 
option may not 
be exercised 
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7.81 Subsequently, as early as 1981, 
the Deputy Chief of the Defence Staff 
addressed a memorandum to the Chief of 
the Defence Staff on the AVGP Trials — 
Cyprus and Canadian Forces Europe, 
which confirmed the Cougar’s 
unsuitability as an operational vehicle. 
The latest document approved by the 
Department in relation to this vehicle was 
in March 1993. It describes how “the 
Cougar was introduced in 1978 as a tank 
trainer and has since been designated as 
an operational vehicle by default... In 
short, the Cougar does not provide an 
acceptable armour capability for low-level 
operations.... If the armour capability for 
low-level operations is not upgraded, the 
CF will not have a general purpose 
combat capability as required. The Land 
Force will be unable to fulfil its assigned 
missions and tasks. The current capability 
deficiency limits Canada’s meaningful 
participation in future coalition and 
peacemaking operations.... The 
replacement of the Cougar with an 
Armour Combat Vehicle is essential if the 


Land Force is to satisfy the ... qualitative 
requirements of low level operations.” 


7.82 Our review of reports from our 
sample of missions tends to confirm the 
documented conclusion that the Cougar is 
increasingly inadequate for the full range 
of peacekeeping missions (see 

Exhibit 7.9). Officials told us that military 
officers so far have successfully managed 
the risk. Our review of casualty reports 
found no direct attribution of deaths or 
injuries to these vehicles. 


7.83 Comparability with other 
nations. The Department conducted a 
comparison of equipment provided to one 
Canadian battalion sent to the former 
Yugoslavia in 1994. The study concluded 
that the Canadian battalion had about 

70 percent of the capability of a 
comparable British battalion in the same 
area with similar tasks. It noted that 
without emergency purchasing, the level 
would have been about 40 percent. The 
study observes that emergency purchasing 
is not the most efficient method of 


Canadian Cougar tank 
trainer employed in the 
Balkans (see 
paragraph 7.80). 
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rectifying deficiencies. Our audit showed 
that, on average, these emergency 
purchases took over 180 days, or one full 
rotation, to reach the field forces. 


7.84 The Department’s study 
compared 48 items of equipment and rated 
each of them as inferior, comparable or 
superior, after allowing for emergency 
buys (see Exhibit 7.10). 


7.85 Among items described as 
inferior were heavy machine gun/light 
cannon, fragmentation vest, light 
anti-armour weapon, individual night 
weapon sights, counter mortar radar, APC 
(tracked), and the Cougar. 


7.86 Our interviews with returning 
Canadian commanding officers revealed 
further inadequacies in night vision 
equipment, global positioning equipment 
and telecommunications equipment in 
Somalia. 


7.87 When considering the use of the 
current APC and AVGP fleets on 
peacekeeping operations, the 
Department should prepare and make 
available to decision makers a fully 
documented assessment of risk. 
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Department’s response: The Department 
concurs with the recommendation. Threat 
assessments are completed as part of the 
planning process for all potential 
Canadian Forces (CF) participation in 
peacekeeping operations. These 
assessments are based on the anticipated 
mission of the CF in the area of 
operations and recognize the capabilities 
and limitations inherent in the force 
structure and equipment of the planned 
CF contingent. The planning process will 
ensure that these risk assessments are 
included in the departmental 
recommendations to government. 


Training 


Training for peacekeeping operations 


7.88 Predeployment training is 
improving, but has often been 
inadequate. The Canadian Forces have 
long maintained that general military 
training is the best preparation for 
peacekeeping duty. More recently, 
depending on the mission, the Canadian 
Forces also conduct mission-oriented 
predeployment training of varying 
complexity. 

7.89 Due to the evolving nature of 


peacekeeping operations, we audited the 
adequacy of predeployment training, 


From Somalia: 7 


mines.” 


recommendations as to its future use in operations.” 


- From UNPROFOR: 
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Exhibit 7.9 

— “Small arms ballistic protection (i.e., up to 14.5 mm proof) element has not been tested in Somalia Vehicle Deficiencies Noted in 
and is considered to be a serious outstanding deficiency in the AVGP.” Operations 

“The Cougar is in service with A Sqn, the RCD in the Matavan area of operations. This is probably 

the most difficult and dangerous area of operations in the Canadian Sector. The Cougar was never 

intended to be used in operations. It is a ‘tank trainer’. Even so it has been deployed to Somalia 

and in the course of operations has been exposed to tank fire, 120 mm mortar fire and antitank 

“Remark; Director, Armour should study the use of the Cougar in Somalia to make appropriate 

“Main threat is from individuals armed with weapons up to 14.5 mm.” 
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Exhibit 7.10 


Departmental Comparison of 
Canadian and Foreign 
Equipment 


including equipment available. We also 
examined predeployment training of the 
Reserves. In addition, we reviewed 
mission records for a selection of 
peacekeeping contingents deployed since 
1992, reviewed equipment records, and 
examined data from Land Force 
evaluations of the Reserves for contingent 
training in 1994 and 1995. We also visited 
training sites and conducted on-site 
interviews. 


7.90 We found that predeployment 
peacekeeping training has become more 
systematic in the last two years. 
Subsequent to a 1992 departmental 
program evaluation, which was critical of 
the ad hoc nature of training for 
peacekeeping, the Deputy Chief of the 
Defence Staff issued a document entitled 
Training Requirements for Peacekeeping 
Operations. This document corrected the 
view that general training was sufficient 
training for peacekeeping. It stated that 
when preparing to deploy, general-purpose 
military training needs to be supplemented 
by specific training for peacekeeping. 
Detailed training guidance, particularly 
for rotations to the former Yugoslavia, 
have been issued. However, at least for the 
last rotations of Operations Harmony and 
Cavalier, the Command-level guidance 
was not disseminated in a timely fashion 
to units, who were therefore forced to 
develop their own training. Some units 
have also commented in post-operation 
reports on the lack of standards 
specifically for peacekeeping training. 


7.91 Peacekeeping experience is not 
always being passed from contingent to 


Category _ Number of Items 
Superior 10 
Comparable pp) 
Inferior 16 


contingent, or among Commands. 
Although Air Command followed 
Training Requirements for Peacekeeping 
Operations in its preparation for the 
mission to Haiti, a post-training report 
stated that training in the Rules of 
Engagement should have been improved 
and noted the training practices employed 
by U.S. forces. Land Force Command 
employs techniques very similar to those 
of the U.S. forces. However, Air 
Command seemed unaware of this. 


7.92 The absence, until recently, of an 
effective “lessons learned” process (only 
now becoming fully effective) in the Land 
Force has meant that some problem areas 
were encountered repeatedly, in spite of 
the cumulative experience of many 
combat arms rotations. Some areas of 
repeated weakness noted by mission 
reports included: training in Rules of 
Engagement, familiarization with foreign 
weapons, first aid training, and 
negotiation training. There is often no 
opportunity to make up for these 
deficiencies once troops are in theatre. 
The work of the Army Lessons Learned 
Centre and the Canadian Forces 
Peacekeeping Support Centre is intended 
to address this problem. The Deputy Chief 
of the Defence Staff has undertaken a 
review to be completed 31 March 1996 to 
identify further improvements. 


7.93 Individuals and contingents are 
sometimes deployed even though they 
have not been fully or consistently trained. 
Several UNPROFOR mission reports 
recommended better standards of 
individual training before deployment as 
well as remedial training in the theatre of 
operations. The rotation contingent to 
Haiti had to conduct further training in 
theatre to make up deficiencies. 


7.94 We found that numbers and types 
of equipment used to train rotation 
contingents for peacekeeping operations 
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do not closely match numbers and types in 
theatre. As well, certain types of 
specialized equipment used in operations 
are not available for training in Canada. 
However, our examination of mission 
reports has not revealed this to be a 
critical problem. 


7.95 The Canadian Forces should 
develop adequate standards for 
predeployment training for 
peacekeeping. They should also ensure 
that improvements in training 
techniques are quickly disseminated. 


Department’s response: The Deputy Chief 
of the Defence Staff has commissioned a 
review of peacekeeping training. 
Following the completion of the review, 
required improvements will be 
incorporated in Course Training 
Standards and will be used in the training 
provided to both individuals and units 
proceeding on peacekeeping duties. 


Coilective training has suffered 


7.96 Peacekeeping operations 
contributed to the decline in Land Force 
collective training. The main training 
exercise for the Land Force 

(Rendez-Vous 95), held only once every 
three years, was cancelled due to 
peacekeeping operations. The 
smaller-scale exercise that was conducted 
instead revealed significant deficiencies in 
collective training for the Land Force and 
in readiness levels. By mid-1994, Land 
Force Command was experiencing what 
was described by the Army Training 
Structure Review as a “collective training 
crisis”. Combined arms training at the 
battle group level and above had come to 
a virtual standstill since 1992, and leaders 
were beginning to rotate through 
commands without having exercised their 
unit or formation. The Army Training 
Structure Review (ATSR) concluded that 
this was unacceptable, since the army 
intended to fight as a combined arms team 
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and must therefore train as such. The 
review team thus judged the army’s 
deployment plan for one brigade to be 
unrealistic. Reports of the Command 
Operational Readiness Effectiveness 
System (ORES) in 1993, 1994 and 1995 
confirmed that collective training was 
severely limited and formation training 
non-existent, undermining readiness to 
conduct formation-level tasks. They 
further identified “UN deployments” as 
one of the causes of the decline in 
collective and formation-level training. 


LD), The Army Training Structure 
Review recommended two command post 
exercises and one brigade field exercise 
every two years. The army chain of 
command directed that one exercise 
would be held every three years, and 
scheduled Rendez-Vous 95. However, this 
exercise was cancelled due to conflicts 
with the preparations for Canadian 
Operation COBRA, which aimed at the 
potential withdrawal under adverse 
conditions of Canadian forces from the 
former Yugoslavia. It was replaced by 
Exercise VENOM STRIKE to prepare for 
that operation. The size and scope of that 
exercise were significantly different from 
those of Rendez-Vous 95. 


7.98 We reviewed the planning and 
implementation of both Rendez-Vous 95 
and VENOM STRIKE to determine what 
direct impact peacekeeping had on the 
exercise-level collective training. We 
wanted to ascertain whether the army can 
conduct formation-level collective 
training to the required standard in the 
face of peacekeeping commitments near 
the deployment goals set in the 1994 
White Paper. We also wanted to observe 
the extent to which the army was 
beginning to address its training crisis. 


7.99 The aim of Rendez-Vous 95 had 
been to validate Land Force Command 
force generation plans for joint and 
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The most recent 
brigade exercise 
completed only 3 of 12 
battle task standards 
and ended with the 
brigade 14 days away 
from being ready. 


combined operations. The aim of VENOM 
STRIKE was to prepare for the specific 
mission to the Adriatic. An important shift 
was made in the training standards to be 
exercised. The collective battle task 
standards at the brigade group level were 
changed with consequent implications for 
the lower-level objectives. 


7.100 There were several changes to the 
exercise plan. The total budget of the 
exercise was reduced from a projected 

$18 million to $6 million. Western Area 
participation was cancelled and Quebec 
and Central Area participation was 
drastically reduced. Participation by Air 
Command and other functional elements 
was also reduced substantially. 


7.101 For the reduced brigade group 
remaining in VENOM STRIKE, as was 
planned for Rendez-Vous 95, only a 
limited number of battle task standards 
were exercised. Achieving 5 of the 12 
standards was designated as an objective 
of VENOM STRIKE, but in the end only 
three were exercised and achieved. 
Exercising only 3 of 12 battle task 
standards in a three-year period means 
that, given command and personnel 
rotation, at no time in the foreseeable 
future would there be any fully trained 
formations, units or even personnel in the 
army. 


7.102 Furthermore, the post-exercise 
report stated, “The most optimistic 
interpretation of readiness, on the last day 
of VENOM STRIKE, was that the land 
component was still 14 days away from 
being ready.”’ Nevertheless, the COBRA 
force was declared operationally ready to 
deploy. Thus the contention that army 
deployment plans are unrealistic seems 
justified. 


7.103 Improvement since 1994. 
According to Chapter 24 of our 1994 
Report, Land Force Command did not 


assess whether exercises met operational 
requirements. Since then, the Land 
Force’s approach has clearly improved to 
become more systematic. Exercise 
VENOM STRIKE had predefined 
standards (the battle task standards), 
against which performance could be 
evaluated. An extensive evaluation plan 
and process was elaborated, drawing 
directly on the battle task standards to 
conduct the evaluation. As well, an 
attempt was made to capture the lessons 
learned from the exercise and feed them 
into a systematic process through the 
Army Lessons Learned Centre. 


7.104 We observed that for the first 
time, handling of civil-military 
co-operation was given some prominence 
in a brigade-level exercise. The army 
exercised on working with 
non-governmental agencies in a 
peacekeeping environment, on refugee 
handling and on civil-military negotiation. 
Many lessons were learned. 


7.105 The exercise concluded that 
much more remains to be done to train 
individual soldiers on Rules of 
Engagement. This was of particular 
concern, given the real-world operational 
focus of the exercise. 


7.106 At the levels below brigade 
group, Exercise VENOM STRIKE 
provided some significant indicators of 
deterioration in the level of collective 
training. The post-exercise report noted 
that one of the two battle groups had 
arrived for the exercise without having 
completed the entry standards, in spite of 
the fact that the exercise allowed 72 days 
for assembly and preparatory training, 
whereas the departmental deployment 
operational plan allows only 29 days. The 
report stated: “This causes grave concern 
in an army that had been told for two 
years that Rendez-Vous 95 was the single 
highest priority event of 1995.” The 
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post-exercise report also highlighted 
weaknesses in the tactical handling of 
armour, which “can be attributed perhaps 
to the lack of training opportunities caused 
by operational and other commitments 
during the past few years.” 


7.107 We examined a number of Land 
Force collective training and exercise 
plans to determine the extent to which 
peacekeeping may be causing the 
cancellation or significant modification of 
planned training exercises. We selected a 
sample of 130 exercises planned between 
1992-1993 and 1994-1995. 


7.108 Operational Readiness and 
Effectiveness System (ORES) and other 
reports for that period noted that the 
tempo of UN operational deployments, 
coupled with funding constraints, 
“impacted heavily” on unit and 
formation-level training. One ORES 
report stated that there was a cumulative 
decay in combat functions not directly 
employed in peacekeeping, and in the . 
ability and readiness of brigades to meet 
formation-level commitments. 


7.109 Of the 130 exercises selected in 
our sample, we could find evidence of 
only 46 that took place. Planned exercises 
involving a battalion or regiment-sized 
unit were the ones most often cancelled, 
with completion rates of less than 

20 percent. Peacekeeping appears to have 
had an impact on one of two documented 
command post exercises conducted, and 
on two of six documented formation-level 
exercises. The fact that only 35 percent of 
planned exercises took place and that 
peacekeeping had a significant impact on 
the more important ones that did take 
place suggests that the training crisis 
identified by Land Force staff extends 
down to the unit level. 


7.110 We examined the equipment 
available for collective training to 
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determine whether operational tasks were 
impeding collective training on this score. 
Exercise documents revealed no systemic 
problems due to lack of equipment for 
training that can be traced to 
peacekeeping operations. Shortages of 
equipment for particular activities are 
normally made up by borrowing from 
other units for the training period. As 
there are no scales for the type and 
amount of equipment required to conduct 
collective training, there is no objective 
standard against which equipment 
available for exercises can be measured. 


7.111 The Land Force should clarify 
its concept of operational readiness. 
Land Force field training regulations 
and standards should be completed as a 
matter of priority and should address 
the links between readiness, resources 
and the training achievement of battle 
task standards. The Land Force should 
address its inability to complete 
collective training plans and the impact 
on collective training of peacekeeping 
commitments. Battle task standards 
should be exercised within a reasonable 
time frame. 


Department’s response: The increase to 
the size of the field force, once fully 
implemented, will improve Land Force 
Command's ability to conduct necessary 
collective training while simultaneously 
conducting peacekeeping operations. An 
additional improvement to collective 
training as a result of the operational 
enhancements will be the reduction of 
turmoil amongst the remainder of the field 
force caused by the current policy of 
drawing on other units to augment 
deploying units. Land Force Command is 
in the process of implementing individual 
and collective battle task standards, field 
training regulations and the necessary 
linkage between readiness, resources and 
training requirements to address this 
deficiency. It must be appreciated that 
there will always be cases where 
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operations will affect the ability to train 
but every effort will be taken to minimize 
the impact on overall readiness. 


Peacekeeping Support 


Preparedness for civil-military 
co-operation 


7.112 The Canadian Forces are 
improving their doctrine and training 
for civil-military co-operation (CIMIC). 
In recent peacekeeping operations, the 
military role has often been to support 
humanitarian activities. In the former 
Yugoslavia, for example, the United 
Nations High Commissioner for Refugees 
(UNHCR) was the lead agency for United 
Nations efforts, not UNPROFOR. Recent 
missions have clearly demonstrated the 
need to establish a sound working 
relationship with the many other players, 
such as United Nations’ agencies, private 
volunteer organizations, local civic 
leaders and diplomatic staff, to address 
effectively problems such as refugee 
handling. Military forces around the world 
have recognized that new doctrine and 
training are required. 


7.113 Britain and the United States 
have been addressing this area more 
actively than have the Canadian Forces. 
They have developed doctrine, and the 
United States has developed a specific 
training exercise. The United States 
deployed to the United Nations mission in 
Haiti members of a Reserve unit that is 
entirely dedicated to the CIMIC role. 


7.114 The Canadian Forces are now 
taking active steps to develop doctrine and 
training for CIMIC. The new 
Peacekeeping Manual addresses the main 
points of CIMIC, and officials told us that 
further details will be added in the coming 
months. As well, the draft Land Force 
doctrine manual clarifies the objectives 
and application of CIMIC and 


humanitarian aid. Contingent commanders 
had expressed the concern to us that they 
did not feel supported by the Canadian or 
United Nations chains of command when 
encouraging humanitarian aid initiatives 
by their troops. The draft doctrine now 
makes it clear that this is a desirable way 
of attaining the overall military objective. 
As noted in Exercise VENOM STRIKE, 
the Land Force is beginning to exercise its 
CIMIC role and is deriving valuable 
lessons. 


7.115 We also found that the doctrine 
being developed incorporates a distinctly 
Canadian approach to peacekeeping. 
Considerable emphasis is being placed on 
the value of recognizing where consent by 
opposing sides will allow the military to 
rely on civilian organizations. At the same 
time, the importance of demonstrating 
impartiality in all humanitarian and 
CIMIC activities is also being 
emphasized. However, during our field 
visits and discussions with returned 
contingent commanders as well as with 
the officers charged with CIMIC roles, we 
noted that they were not fully aware of the 
ongoing work on doctrine development. 


7.116 The Canadian Forces should 
ensure that doctrine addressing CIMIC 
is finalized quickly and is implemented 
through appropriate training in all 
Commands. 


Department’s response: The Department 
concurs with the recommendation. 
Development of CIMIC doctrine will be 
pursued as a priority and the Deputy 
Chief of the Defence Staff directed that 
peacekeeping training will be modified to 
include training in CIMIC. 


Control of inventory and equipment 


7.117 Some important Canadian 
Forces Supply System controls have 
broken down. To determine whether 
supplies and equipment to Canadian 
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contingents on peacekeeping missions 
were provided with due regard to 
economy and efficiency, we reviewed a 
departmental database of items sent to 
missions between September 1991 and 
June 1995. The database recorded 

$482 million in inventory sent, including 
ammunition, spare parts, repairable 
components and consumable items, and 
$340 million in equipment (including the 
recorded value of vehicles), for a total of 
$822 million. 


7.118 We found a clear lack of 
inventory control on items sent to a 
mission. Of 300 sea containers sent to the 
mission in Somalia, many had to be 
opened to determine their contents due to 
lack of proper identification. A 
departmental “lessons learned” report on 
the Somalia mission observed that there 
was inadequate attention to inventory 
control for shipments. The introduction by 
the Department of an automatic tracking 
system is expected to prevent 
reoccurrence of similar problems. 


7.119 | The Canadian Forces do not have 
standard scales for supplies and equipment 
sent to peacekeeping missions. Of the 
$822 million sent, $151 million in stock 
items and $146 million in equipment are 
recorded as still held in theatre. 
Departmental records show a type of 
downward adjustment totalling 

$67 million described as “not affecting the 
write-off register.” They also show 

$13 million of losses due to fire, theft, or 
disposals in theatre. 


7.120 Weconducted more detailed tests 
on these adjustments totalling $80 million. 
We selected a sample that included all 
losses or accounting adjustments greater 
than $100,000, and an additional 30 items 
at random. We requested the 
documentation required by the 
departmental Supply Manual. To the end 
of our audit, the Department could not 
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find a significant proportion of the 
documentation, and the losses and 
adjustments remain largely unexplained. 
For example: 


e Of 31 vehicles recorded as issued to 
the mission in Haiti, 29 were recorded as 
received. Departmental staff recorded in 
the system a loss of two vehicles, without 
providing a documented explanation as 
required. Officials told us they believe 
that the issuing section must have made a 
clerical error and that there was no real 
loss. 


e Another item in Haiti was adjusted 
by more than $11 million. The 
documentation provided to us did not 
identify the issuing supply section. The 
adjustment was not supported by any 
explanation and, contrary to established 
procedures, was unsigned. Although the 
quantity of the item involved makes it 
unlikely that the number stated was 
actually ever on hand, and we believe this 
was a Clerical error and no real loss 
resulted, the failure of the control system 
is troubling. 


7.121 We conclude that the Department 
has failed to apply the required control 
over items flowing through the 
departmental supply system to 
peacekeeping missions. Adjustments 
assumed to correct clerical errors could in 
fact represent losses. This places 
departmental assets at considerable risk. 


7.122 Control procedures to ensure 
that inventory supplied to missions is 
adequately accounted for should be 
communicated well throughout the 
Department and followed. 


Department’s response: The Department 
agrees with the recommendation. 
Improvements to the Canadian Forces 
Supply System and the establishment of a 
central support unit in Montreal for 
deployed operations will improve 
inventory accounting. The introduction of 
new courses for support officers this past 
year has enhanced their ability to control 
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inventory and finances. As part of the 
more comprehensive individual and unit 
training being developed, the importance 
of, and procedures for, inventory control 
will be taught to all personnel prior to 
their deployment on missions. In the 
meantime, incidents noted in this chapter 
are being investigated further by the 
Department. 
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% About the Audit 


Scope 


Our audit focussed on National Defence’s capacity to generate forces for peacekeeping, as well as on whether 
peacekeeping operations are impairing overall training and readiness: whether the troops sent to peacekeeping 
have adequate equipment and are properly taken care of before, during and after deployment; and whether 
peacekeeping operations are costing National Defence more than is necessary. We based our audit primarily 
on the study of six peacekeeping missions: Angola, Haiti, Rwanda, Somalia and the former Yugoslavia 
(UNPROFOR I and UNPROFOR II). 


Although we observed the successes of Canada’s peacekeeping operations in the field, we did not attempt to 
evaluate field operations where Canada’s forces were under the operational control of the United Nations or 
other joint force. 


In the last five years, the involvement of the Canadian Forces in peacekeeping or “peace support operations” 
has greatly expanded. The conflicts in which the Canadian Forces have been involved have often been 
complex and of undetermined duration. Our use of the term “peacekeeping operations” in this chapter 
includes most “operations other than war”. Such operations are characterized by impartiality, the use of force 
only in self-defence and some level of consent of the belligerents to the presence of the military force. 
Peacekeeping operations are difficult and often dangerous because they involve the deployment of Canadian 
Forces personnel into war zones or areas where a cease fire agreement has only recently been signed and the 
parties are still hostile to each other. Although the Canadian Forces participated in many peacekeeping 
missions under United Nations auspices, other missions were part of a multinational combined force. 


Objectives 

‘The overall objective of our audit was to assess the quality of management practices rather than to assess 

military judgments or achievements. Our specific objectives were: 

e to assess the extent of military advice provided to ministers and parliamentarians about Canada’s 
participation in peacekeeping operations; 


e to express an opinion on whether National Defence can meet government commitments for peacekeeping 
without impairing its ability to meet other commitments; 


e to assess whether National Defence’s participation in peacekeeping respects the Canadian Forces’ stated 
values in the treatment of its members and its own personnel policies; 


e to assess whether peacekeeping operations are carried out with due regard to economy and efficiency; 
and 


e _ to describe the nature and extent of the Canadian military contribution to peacekeeping. 
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Criteria 


In assessing the Department, we used goals and objectives established by the government in the 1994 Defence 
White Paper: NATO standards; and military goals and standards established by the Department itself. Detailed 
criteria are discussed in each section along with the audit findings. 


Audit Team 


Francesa Adibe 
Robert Anderson 
Pierre Hamel 

Paul Morse 

Amy Mullins 
Maria Pallante 
Romeo Pasqualitto 
Odette Prégent 


For information, please contact Peter Kasurak, the responsible auditor. 
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Chapter 8 


Canadian Security Intelligence 
Service 


National Headquarters Building Project 


The audit work reported in this chapter was conducted in accordance with the 
legislative mandate, policies and practices of the Office of the Auditor General. 
These policies and practices embrace the standards recommended by the Public 
Sector Accounting and Auditing Board (PSAAB) of the Canadian Institute of 
Chartered Accountants. : 
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Main Points 


Introduction 


Need for a New Facility 
National Headquarters Project Objectives 


Need for Security and Functionality 
Observations and Recommendations 


Schedule and Costs 
Unit cost was appropriate for a special-purpose building 


Security 
The building’s security systems are adequate, but “island” site must be maintained 


Constructed Space 
Program space constructed as authorized 
Constructed gross area significantly exceeds plan, but within the original budget 
A building’s gross area needs to be monitored by the Treasury Board 


User Requirements 
National headquarters meets users’ requirements now and provides for the future 


Contracting Process 
Government contracting regulations and policies were respected 
Redesign of the sixth floor for $1.4 million to accommodate changes to the executive offices 
adds limited functional value 
Consultant fees nearly doubled 
Prepayment of fees contrary to the Financial Administration Act 
Contract form used was inadequate for the purpose 


Options Analysis and Site Selection 
Requirements under the government’s land transfer policy are unclear 
Debate over land transfer results in delays 


Project Review Process 
Reviews of design should be better co-ordinated 
Life-cycle costing analysis was not done, but a value engineering study identified savings 


Opportunities for Cost Savings 
Greater emphasis on cost-saving opportunities is needed 
Approved inflation allowance provides a budget cushion 
Conclusions 
A well-managed project overall with useful lessons learned 


User’s leadership role a key success factor 


About the Audit 
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National Headquarters Building Project 


Assistant Auditor General: David Rattray 
Responsible Auditor: Reno Cyr 


Main Points 


8.1 Since its creation in 1984, the Canadian Security Intelligence Service (CSIS) had needed a headquarters 
facility separate from its predecessor, the former Royal Canadian Mounted Police Security Service. 


8.2 The completion of the new national headquarters building in 1995 has allowed CSIS to consolidate its 
headquarters and Ottawa Regional Office activities from ten separate facilities into one facility. 


8.3 In our opinion, this project was well managed overall, as the national headquarters project was completed 
on time and within budget, and meets the functional needs of its users. 


8.4 There are several success factors or lessons learned in this project that would benefit similar construction 
projects; one of those factors is the key role played by CSIS as the building’s user. 


8.5 We believe that the project team took appropriate steps to ensure that the completed facility provides a 
secure environment for CSIS employees and their activities. 


8.6 The fact that the constructed gross area exceeded the original planned gross area by over 13,000 square 
metres, without exceeding the approved budget, raises concerns about how project budgets are calculated and 
about the effectiveness of Treasury Board control over its project approvals. 


8.7 The redesign of the sixth floor, which was done despite the concerns of the project team and at an 
additional cost of approximately $1.4 million, did not, in our opinion, significantly improve the functionality of 
the building. 


8.8 We also noted contract prepayments made in advance of work, contrary to the Financial Administration 
Act, as well as inadequate wording in the contract documents for the architect, resulting in reduced accountability 
and increased costs. 


8.9 We noted examples where, although no authorities or budgets were exceeded, additional funds could 
have been saved on this project. 
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Introduction 


Need for a New Facility 


8.10 The Canadian Security 
Intelligence Service (CSIS) was created 
on 16 July 1984, replacing the Royal 
Canadian Mounted Police (RCMP) 
Security Service. As part of the transition 
to a civilian agency, physical separation 
from the RCMP was considered essential. 
The accommodation problem was 
examined in a 1987 Report to the Solicitor 
General by the Independent Advisory 
Team on the Canadian Security 
Intelligence Service, entitled “People And 
Process In Transition’, and also known as 
the Osbaldeston Report. 


8.11 The Osbaldeston Report 
recommended, “An immediate solution to 
the CSIS accommodation problem must 
be found, bringing all headquarters 
functions together in a building that can 
be adapted to meet the needs of CSIS, 
both human and technological.” 


8.12 In 1987, CSIS headquarters was 
occupying eight separate facilities at six 
locations scattered throughout the 
National Capital Region. In addition, its 
Ottawa Regional Office was located in 
two separate facilities. 


8.13 Compounding the 
accommodation problem was the need for 
expanded computer facilities. The CSIS 
computer centre had been co-located with 
that of the RCMP. The CSIS electronic 
data processing requirements were 
growing and more space was needed, 
which was not available in that location. 
The RCMP also wanted to expand its 
computer operations and needed the space 
that CSIS was occupying. 


8.14. = The Treasury Board’s approval of 
the project was based on the 
understanding that the site and building 
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would be designed to provide sufficient 
flexibility to accommodate growth for the 
foreseeable future. 


8.15 Cabinet approved the 
construction of a new headquarters 
complex for CSIS in September 1988. 
(See Exhibit 8.1 for the chronology of key 
project events.) 


As part of the 
transition to a civilian 
agency, physical 
separation from the 
RCMP was considered 
essential. 


8.16 CSIS plans in 1989 called for 
building the national headquarters 
complex in two phases. Phase I comprised 
the computing centre and utilities 
building, targeted for completion in 
August 1991. Phase II comprised the 
balance of the complex, to be completed 
in July 1994. 


8.17 In December 1989, the 
government-imposed construction freeze 
delayed the start of Phase II. However, as 
Phase I was already started and considered 
necessary to maintain essential services, 
the freeze did not affect it. Subsequent 
approval was given to start Phase II 
foundations in November 1991. As a 
result of the early start of Phase II 

(five months before the construction 
freeze expired), the Treasury Board pulled 
back $1.5 million of inflation allowance. 


8.18 During construction of the 
national headquarters facility, CSIS was 
subject to the same budget cutbacks 
experienced by all government 
departments. The cutbacks resulted in a 
significant reduction in CSIS headquarters 
staffing levels, creating unforeseen surplus 
office space in the new facility. CSIS 
responded by relocating its Ottawa 
Regional Office operations into the 
facility, and adjusting the project scope for 
special regional needs. 


8.19 The building was completed in 
February 1995, with a gross area of 
64,293 square metres and a total project 
cost of approximately $153.5 million. 
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National Headquarters Project 
Objectives 


8.20 CSIS identified key objectives for 
its headquarters facility project 
(Exhibit 8.2). 


Need for Security and 
Functionality 


8.21 CSIS activities revolve around 
information that, in accordance with the 
Canadian Security Intelligence Service 
Act and the Government Security Policy, 
is accessible only on a “need to know” 
basis. As a result, office areas must be 
compartmentalized to a greater extent 
than is found in typical office buildings. 
This results in a slightly larger than 
normal area of office space per occupant. 


However, the individual offices adhere to 
the Treasury Board administrative 
guidelines for office accommodation. 


8.22 CSIS required that the concepts 
used in the design of the complex reflect 
the need for modularity, flexibility and 
expandability. Due to the restrictions 
imposed on the exterior of the building for 
such essential items as windows, CSIS 
stated that the complex could incorporate 
one or more internal courts or atria. 


8.23 A requirement of the design was 
to build flexibility into all space to 
accommodate organizational changes. An 
important element in the flexibility 
requirement was the need to provide for 
secure data communication services at 
every work station in the office areas. 


Exhibit 8.1 


Chronology of the Key Events 
of the National Headquarters 
Project 


1988 


1989 


1990 


1991 


1993 


1995 


In September, Cabinet approved a recommendation for a new headquarters 
complex for CSIS. 


In April, the Treasury Board gave preliminary project approval for 
$142.4 million (current year dollars) to construct the national headquarters 
building. It was designated as a major Crown project, with the additional 
monitoring and reporting requirements that entails. 


In December, the government imposed a construction freeze on all major 
construction projects (Phase I was already under way). 


In March, the Treasury Board increased the project funding under 
preliminary project approval to $152.5 million to cover additional costs 
related to the construction freeze. 


The Treasury Board also gave effective project approval to construct Phase I 
at an estimated cost of $43.65 million, with the provision that any unused 
balance could be carried forward to Phase II. 


In August, the Treasury Board gave effective project approval for Phase IL, 
with a total estimated cost of $113.3 million. 


Phase I was completed, at a cost of $37.7 million, and the balance of funding 
of $5.9 million was carried forward to Phase II. The project scope for Phase II 
was increased by adding a fifth floor to the south tower to provide flexibility 
in the form of unfinished floor area for future expansion. 


In December, the Treasury Board approved a revised effective project 
approval for Phase II, which included an addition to accommodate the 
Ottawa Regional Office, at an additional cost of $2.5 million. 


In February, CSIS took possession of the building. In November, it 
completed the relocation of all its employees. 
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8.24 Although CSIS stated that the 
building should be of a quality similar to 
that of any national headquarters, security 
and functionality were its prime objectives 
for this project. Also specifically 
mentioned was the requirement to deliver 
the project on time and within scope and 
budget. 


Observations and 
Recommendations 


8.25 The objective of this audit was to 
determine whether the constructed 
national headquarters facility met the 
CSIS-stated objectives and the Treasury 
Board approvals, and whether the project 
was implemented with due regard to 
economy and efficiency. We examined all 
the main stages of this major Crown 
project. Further details on audit 
objectives, criteria, scope and approach 
can be found at the end of the chapter in 
the section “About the Audit”. 


Schedule and Costs 


8.26 The project was completed 
within the approved time frame, in 
February 1995. The original completion 
target of July 1994, at preliminary project 
approval, had been revised after the 
government-imposed construction freeze 
was expected to delay completion by 
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17 months. Once work on Phase II had 
begun, however, the schedule was again 
brought forward, with a new completion 
target of September 1995. 


Unit cost was appropriate for a 
special-purpose building 


8.27 Based on unaudited final costs, it 
is estimated that the total project was 
completed for approximately $500,000 
less than the revised budget of 

$153.5 million. Final costs cannot be 
determined until all invoices and changes 
have been verified and settled. 


8.28 The unit costs for this building 
are higher than those for typical office 
buildings, because of the need for 
specialized facilities such as wet and dry 
labs, workshops, and special security 
requirements and considerations that 
created extra procedures and costs during 
the construction phase. In our opinion, 
however, the unit costs for this project 
were appropriate. (See Exhibit 8.3 for 
space allocation.) 


Security 


The building’s security systems are 
adequate, but “island” site must be 
maintained 


8.29 A primary requirement of this 
facility is that it be a secure environment. 
We reviewed the threat and risk 
assessments, the security recommenda- 


The project objectives were to create: 


storage and administrative offices. 


@ a secure, technologically advanced environment in a structure that will be able to 
respond to evolving accommodation needs both easily and economically; 


@ a radiation—-shielded computing centre that will protect the vital database of CSIS 
and provide for future expansion in this area; 


© space for future employee growth until 2004, based on an assumed one percent per 
year growth in national headquarters personnel (subsequently deleted); and 


@ acafeteria, day-care centre (subsequently deleted), training rooms, workshops, 


It is estimated that the 
total project was 
completed for 
approximately 
$500,000 less than the 
revised budget of 
$153.5 million. 


Exhibit 8.2 


National Headquarters Project 
Objectives 
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The security design is 
integral to the building 
and has been part of 
the overall design from 
the beginning. 


Exhibit 8.3 


tions and the security design. We also 
reviewed the security precautions and 
measures taken during construction, and 
interviewed experts in the security field 
from other government departments. We 
concluded that the project team took 
appropriate steps to ensure that the 
completed facility provides a secure 
environment for CSIS employees and their 
activities, without being onerous. 


8.30 The security design is integral to 
the building and has been part of the 
overall design from the beginning. This 
approach has minimized the cost of 
installing additional special security 
measures. The security system can operate 
at different levels. Should a situation 
develop where CSIS would require 
heightened security, overall or in any part 
of the building, the system can 
accommodate such changes. It has also 
been designed to incorporate new 
technology easily. 


8.31 In our opinion, the nature of the 
isolated site on which this facility is 
situated is a key security factor in the 
design. The site uses physical space 
surrounding the building as a protective 
buffer. It is therefore imperative that CSIS 
ensure the integrity of this “island” site. 
Should adjacent lands be developed, some 
of the design precepts would be rendered 
invalid. 


Office and Storage 


Allocation of National 
Headquarters Space 


Source: CSIS 
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(65.3%) 


Common Service 
(13.3%) 


Special Purpose 
(21.4%) 


Constructed Space 


Program space constructed as 
authorized 


8.32 Our audit examined whether the 
amount of space constructed is consistent 
with that authorized by the Treasury 
Board. We concluded that the program 
space is consistent with that authorized, 
but that the gross area significantly 
exceeds that proposed by CSIS in the 
preliminary project approval. 


8.33 A project is initially defined in 
terms of its scope, including program 
space, schedule, and budget. Program 
space is a value based on the user’s 
statement of requirements, and may be 
considered the minimum amount of space 
required for an organization to meet its 
operational needs. Once approved by the 
Treasury Board, program space becomes a 
control point against which performance is 
measured. For the completed national 
headquarters building, the program space 
is consistent with the 38,410 square 
metres approved by the Treasury Board. 


Constructed gross area significantly 
exceeds plan, but within the original 
budget 


8.34 In the preliminary project 
approval, the building’s planned gross 
area of 50,836 square metres was 
calculated by applying a design “net to 
gross” factor to the program space. The 
project budget was then determined by 
multiplying the calculated gross area by 
the estimated cost per square metre, using 
industry standards. However, the gross 
area is not included in the project scope 
and it is not formally approved or 
monitored by Treasury Board. 


8.35 We noted that the constructed 
gross area exceeds the originally 
estimated requirement by over 

13,000 square metres. As early as 
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February 1992, the progress reports to the 
Treasury Board Secretariat were showing 
a constructed gross area of 64,300 square 
metres, and we were advised that the final 
constructed gross area is 64,293 square 
metres. Part of the increase in the gross 
area can be accounted for by the 
unfinished fifth floor (for future 
expansion) and the Ottawa Region space, 
both of which were approved after the 
preliminary project approval. In addition, 
the original gross area was estimated 
before the design had been started. 
However, despite being made aware of 
this 26 percent increase in gross area, we 
Saw no indication that Treasury Board 
officials communicated concern to the 
project leader. 


8.36 It is also important to note that 
the increase of over 13,000 square metres 
in gross area was accomplished with no 
increase in the project budget. This major 
change in the gross area, with no 
corresponding change in the budget, raises 
concerns about how project budgets are 
calculated, since the amount of area built 
relates directly to the total cost. When 
such changes are proposed, the 
assumptions used in estimating the budget 
should be verified. 


A building’s gross area needs to be 
monitored by the Treasury Board 


8.37 Using program space, budgets 
and schedules as the primary means of 
controlling projects does not guarantee the 
best value for the taxpayer. Since the gross 
area is fundamental to the cost of a 
building, it should also be included as part 
of the approved project scope. Any 
proposed changes to the gross area would 
have to be explained, thereby giving the 
Treasury Board a more complete 
understanding of what is happening in the 
project. 
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8.38 The Treasury Board Secretariat 
should review the assumptions used by 
departments in developing project 
budgets and the current mechanisms for 
controlling and monitoring the scope in 
proposed construction projects. 


User Requirements 


National headquarters meets users’ 
requirements now and provides for the 
future 


8.39 Based on interviews with CSIS 
employees, we are satisfied that the 
national headquarters facility meets the 
requirements of its users in day-to-day 
operations. The building provides a 
functional workspace and a productive 
working environment, reflecting the great 
importance that CSIS placed on functional 
requirements. 


8.40 During February 1989, the initial 
user program space requirements were 
subjected to a rigorous analysis by the 
CSIS project team, using a 
challenge-review process. This resulted in 
significantly reduced program space. CSIS 
stated to the Treasury Board that, as a 
result of this analysis, the user program 
space incorporates only those needs that 
are the minimum acceptable for 
operational purposes while still adhering 
to the project objectives. 


Since the gross area is 
fundamental to the 
cost of a building, it is 
important that it be 
included as part of the 
approved project 
scope. 


8.41 The analysis also determined that 
the reception/dining room, located next to 
the corporate boardroom, could be 
eliminated. Despite this recommendation, 
the reception/dining room space was 
constructed. Although we did not see a 
justification of the need for this space as 
part of the analysis, CSIS has stated that it 
needed the reception/dining room for 
entertaining visitors privately. 


8.42 The building’s design 
incorporates flexibility for future change. 
For example, the infrastructure for data 
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We determined that 
the contracts and 
change orders were 
administered in 
compliance with the 
government 
contracting 
regulations. 


The National 
Headquarters 
Building of CSIS 


and electrical services in each workspace 
has been designed to allow for future 
growth. Furthermore, the building’s 
systems have been constructed to be 
adaptable in a changing technological 
environment. 


8.43 Our audit also assessed whether 
the facility exceeds user requirements. We 
concluded that in most cases it does not. 
We noted that the project team reviewed 
all requirements for reasonableness and 
functionality. The design was frozen and 
changes to user requirements were not 
allowed after a specified time (an 
exception is the redesign of the executive 
suite, which is discussed under 
“Contracting Process”). 


8.44 Since the start of construction, 
CSIS has been subject to government 
budget cuts and staff reductions. Despite 
the subsequent relocation of the Ottawa 
Regional Office activities into the national 
headquarters facility to utilize some of the 
excess space created by staff reductions, 
there remains some underutilized office 
space. CSIS has advised other agencies 
that it might have space available. We 
encourage this action. 


8.45 CSIS should conduct a space 
utilization study to determine how much 
surplus space is available, and continue 
to examine alternatives for its efficient 
use without compromising its security 
requirements. 


Contracting Process 


Government contracting regulations 
and policies were respected 


8.46 Public Works and Government 
Services Canada (Public Works) acted as 
the contracting authority for the project. 
We examined a sample of the construction 
contracts, construction change orders, 
consultant contracts and fees paid to the 
Department. Changes are inevitable in any 
construction project, and a provision for 
them is always included in a contingency 
allowance. 


8.47 We determined that the contracts 
and change orders were administered in 
compliance with the government 
contracting regulations, and with policies 
and procedures set out for the project. The 
changes were justified, based on 
functional requirements, and generally 
represented due regard to economy. 
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Redesign of the sixth floor for 

$1.4 million to accommodate changes to 
the executive offices adds limited 
functional value 


8.48 We noted one change, however, 
that did not represent good value for 
money. The change was the redesign of 
the sixth floor, which contained the 
executive offices, and it took place after 
other design changes had been frozen. 
Although the project team had concerns 
about the cost and the effect on the 
schedule, it was decided in January 1993 
to redesign the sixth floor completely as a 
result of changes to the location and 
layout of the executive offices, to make 
other functional changes and to add doors 
to a rooftop terrace. In our opinion, this 
work does not appreciably increase the 
functionality of the space. We estimate 
that the work cost approximately 

$1.4 million to complete. However, it is 
important to note that no completed work 
had to be undone as a result of the 
changes since they were approved before 
work commenced. This was the only such 
case that we noted. 


CSIS comments: In reference to item 
8.48, CSIS remains convinced that the 
revisions to the sixth floor represent good 
value. The executive offices are much 
more accessible in the current 
arrangement and additional functions 
were introduced. 


Consultant fees nearly doubled 


8.49 Consultant fees were estimated in 
the preliminary project approval to be 
approximately $8.75 million. Final 
consultant fees were $15.4 million. Public 
Works explained to the Treasury Board 
that the difference was due to the 
increased use of specialists by CSIS, the 
delay caused by the construction freeze, 
higher than anticipated expenses (for 
example, travel, printing, and translation), 
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additional site supervision, and design 
changes and studies requested by CSIS. 


8.50 The consultant’s contract was 
between Public Works and the 
architectural firm. Public Works selected 
the architects using its established 
evaluation method. Because of the 
selection method (carried out by a 
committee) and the lack of documents on 
file, we are unable to comment on the 
suitability or fairness of the process. 
However, we did note the project team’s 
high degree of satisfaction with the 
architects’ work. Proposed base fees were 
only a minor factor in the selection 
process and were not negotiated. Public 
Works informed us that professional fees 
are now negotiated. 


8.51 The preliminary project approval 
submission estimated that fees and 
expenses paid to Public Works would be 
approximately $3.18 million; instead, they 
were $8.07 million. The increase was 
explained as being due to the inclusion of 
seconded staff, the delay caused by the 
construction freeze, and greater than 
anticipated project complexity. 


8.52 Despite our extensive discussions 
with officials, it remains unclear to us why 
the fees were so much higher than original 
estimates, particularly when other costs 
were lower than expected. In our opinion, 
insufficient control was exercised in the 
management of fees. Had the construction 
contracts come in at the anticipated 
amounts, the project could not have 
incurred these additional expenses without 
exceeding its budget. 


Prepayment of fees contrary to the 
Financial Administration Act 


8.53 We also noted that, in the years 
1989-92, prepayment of Public Works 
fees at year end was made to manage cash 
better, and to avoid lapsing of 
appropriations or reprofiling of reference 


Insufficient control 
was exercised in the 
management of fees. 
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levels of the annual budget (seeking 
approval of the Treasury Board to move 
funds into other fiscal years). This 
prepayment in advance of work was 
contrary to section 34 of the Financial 
Administration Act. 


Contract form used was inadequate for 
the purpose 


8.54 The contract used was the Public 
Works standard contract with generic 
terms and conditions (project-specific 
details are listed in appendices). Given the 
size and complexity of this project, we 
believe it would have been more 
appropriate to use a contract specifically 
written for the national headquarters 
project. For example, under the terms 
governing site supervision, the contract 
states that the architect shall “make visits 
to the site to determine, on an adequate 
sampling basis, whether the work is in 
general conformity with the construction 
documents.” 


8.55 Site supervision is classified as 
an additional service and is extra to the 
basic fee. The contract wording is vague 
and provides no basis for estimating actual 
site supervision requirements, especially 
since “‘adequate sampling basis” and 
“general conformity” are not defined. As a 
result, inadequate provision was made for 
this function and site supervision costs had 
to be increased from $1.35 million to 
$2.32 million. 


8.56 We also noted that the Request 
for Proposal, sent to the architects for the 
selection process and containing the terms 
and conditions of engagement, is not 
referenced in the contract and does not 
form part of the contract. As an example, 
clauses for travel expenses shown in the 
Request for Proposal had limitations and 
restrictions that were not reflected in the 
signed contract. Furthermore, we noted 
that relocation expenses for the consultant 


were paid. Such payment was not 
authorized in the contract. 


8.57 Public Works and Government 
Services Canada should ensure that 
consultant contracts are tailored to the 
specific project. 


8.58 Public Works and Government 
Services Canada should review all fee 
payments to determine that they were 
authorized and appropriate. 


Options Analysis and Site 
Selection 


8.59 Public Works carried out an 
analysis of potential options for meeting 
the CSIS accommodation requirement. 
The analysis determined that it would cost 
more to renovate and add to the Tupper 
Building than to build a new facility, and 
it was therefore decided to request 
approval for a built-for-purpose facility. 
We did not see any evidence that other 
existing buildings, either in the Public 
Works inventory or the private sector, 
were considered. 


Requirements under the government’s 
land transfer policy are unclear 


8.60 Once the decision was made to 
build, six sites were considered. Based on 
the site selection criteria, the Gloucester 
site was chosen as the most appropriate. 


8.61 The site was acquired in two 
stages. The original 12.15 hectare 

(30 acre) site was given to CSIS by 
another government agency. In return, 
CSIS paid $300,000 to cover the costs of 
relocating a microwave tower. Original 
estimates for this work were $150,000. We 
were not able to determine if the increase 
in costs was justified, particularly since 
the new tower has yet to be constructed. 


8.62 Land adjacent to and surrounding 
the original site was subsequently declared 
surplus by the other government agency 
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and acquired by CSIS. CSIS was 
interested in the land to add to and protect 
the security perimeter around the national 
headquarters building. Although the 
Treasury Board submission stated that 
there would be no financial consideration 
involved in the land transfer, we noted 
that CSIS was required to pay an 
additional $250,000 toward the removal of 
the tower. 


Debate over land transfer results in 
delays 


8.63 We also noted that it took more 
than four years to effect the land transfer. 
This delay, and its associated costs, 
resulted from a debate over whether CSIS 
should purchase the land from the 
government. As part of this debate, Public 
Works had the land evaluated by a private 
real estate appraiser. 


8.64 CSIS argued that, at the time of 
the transfer of the original parcel of land, 
the second parcel was under the control of 
the government. However, once the 
second parcel was declared surplus, it 
could be sold. This raised the possibility 
of construction adjacent to the national 
headquarters and thus a compromise of 
the security afforded by the island site. 
The Treasury Board agreed with CSIS and 
the transfer of the second land parcel took 
place. It is not clear why the transfer took 
so long or what benefit the taxpayer 
derived from this process. 


8.65 The Treasury Board Secretariat 
should review the transactions relating 
to the acquisition of the land required 
for the CSIS headquarters project to 
determine if they are in compliance 

with the granted approvals and if there 
are means to streamline such 
transactions. 
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Project Review Process 


Reviews of design should be better 
co-ordinated 


8.66 As part of the design process, 
different groups carried out reviews of the 
building’s design. Preliminary reviews 
were done by the project team, and CSIS 
senior management then reviewed the 
final drawings. The exterior design and 
landscaping were approved by the 
National Capital Commission, following 
review by its Advisory Committee on 
Design in accordance with the provisions 
of the National Capital Act. The Public 
Works Design Review Committee 
reviewed major aspects of the design, 
which also included the exterior of the 
building. 


It took more than four 
years to effect the land 
transfer. 

8.67 Public Works defines its role as 

conducting a peer review of the design. 

This review is carried out as part of its 

responsibility as contracting authority 

with the design consultant. 


8.68 The National Capital 
Commission’s role, as defined by its Act, 
requires the Commission to approve all 
federal construction in the National 
Capital region, in terms of the exterior and 
landscaping elements. In this project, the 
Commission feels strongly that the 
approval process positively influenced the 
design to ensure that it made an 
appropriate contribution to the Capital. 
Furthermore, the Commission informs us 
that it has learned from the experience 
with the CSIS building and has increased 
co-ordination and co-operation on many 
subsequent projects. The Commission also 
states that since 1990 it has further 
streamlined its design review process. 


8.69 We noted the frustrations 
experienced by the project team with this 
review process. There was a general 
feeling that the process was onerous, and 
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A value management 
study was undertaken 
for each of the two 
phases of the building 
project. 


We have entered a new 
era in which there is a 
greater public 
expectation that 
governments will 
exercise every 
opportunity to achieve 
cost savings in 
delivering their 
programs. 


could potentially have resulted in 
extensive delays with marginal benefits to 
the project. 


8.70 
noted a significant variance of opinion on 

the efficiency of the review process. As a 

result, we will examine the review process 
in our future audits of major Crown 


Based on our examination, we 


projects. 


Life-cycle costing analysis was not done, 
but a value engineering study identified 
savings 


8.71 Another kind of design review 
that was performed is usually called value 
engineering or value management. A 
value management study was undertaken 
for each of the two phases of the building 
project. This is the first such study in a 
major Crown project that we have noted. 
The total cost for both studies was 
approximately $150,000. The suggested 
improvements were estimated to provide 
significant savings over the life of the 
building. For example, main components 
of the ventilation system were 
decentralized to each floor, allowing for 
the installation of a more efficient and 
flexible air handling system with 
increased back-up capability. 


8.72 Although value management 
studies are a form of life-cycle costing 
analysis, full life-cycle costing analyses, 
as stipulated in the Treasury Board policy 
for major Crown projects, were not done. 
Furthermore, in the post-mortem of the 
value management studies, it was noted 
that the exercise would have been more 
effective if life-cycle cost data had been 
available. The project team stated that, in 
its opinion, life-cycle costing is not as 
useful as value management. 


8.73 In our experience, life-cycle 
costing (with the exception of the 
Department of National Defence) is not 


done in major capital construction 
projects. We believe that some form of 
analysis of the design and major systems 
should be carried out to ensure best value. 


8.74 While we support the concept 
of project and design review, the 
Treasury Board Secretariat should 
examine the use of value management 
studies versus life-cycle costing studies. 


Opportunities for Cost Savings 


Greater emphasis on cost-saving 
opportunities is needed 


8.75 The ultimate goal of existing 
government policies for project 
management is that budgets not be 
exceeded. We believe, however, that the 
realities of the fiscal restraint climate 
under which we now operate will 
necessitate a change of thinking on the 
part of those approving and managing 
large government expenditures. We have 
entered a new era in which there is a 
greater public expectation that 
governments will exercise every 
opportunity to achieve cost savings in 
delivering their programs. 


8.76 For this project, CSIS was 
operating under the approval of a total 
budget and felt that what it did was within 
the authority granted. We concur; 
however, we believe this situation 
exemplifies the need to clarify what the 
approval limits really mean. Our audit 
identified examples of opportunities 
where, although no authorities or budgets 
were exceeded, additional funds could 
have been saved on this project (see 
paragraph 8.27). 


Approved inflation allowance provides a 
budget cushion 


8.77 The revised preliminary project 
approval in 1991 provided project funding 
of $129 million in constant dollars and 
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$152 million in current (inflated) dollars. 
In other words, there was a provision of 
$23 million for inflation. The provision of 
an inflation allowance is normal practice 
for construction projects that take several 
years to complete. The project budget is 
first calculated in terms of actual 
purchasing power of the dollar in the year 
the budget is approved. Construction costs 
are estimated based on prevailing prices 
for materials, labour and consulting fees 
to arrive at the constant dollar budget. A 
forecast of the expected rate of inflation 
over the period of construction is then 
applied to the constant dollar budget year 
by year to yield the current dollar budget. 


8.78 The actual rate of inflation over 
the life of the project was much lower 
than that assumed in preparing the initial 
project budget. Consequently, the 

$23 million inflation allowance provided 
for in the budget was higher than needed. 
Based on the actual rate of inflation for 
the period 1989 to 1995, we estimate that 
the inflation allowance was approximately 
$10 million to $12 million higher than 
needed. 


8.79 The Treasury Board effectively 
signalled the notion that the inflation 
allowance is an “as required” cost element 
in the project budget, when it recovered 
$1.5 million from the budget in 1991 
because the project was able to begin 
sooner than anticipated. 


8.80 The inflation allowance provided 
a “cushion” in the budget, and this 
cushion, in addition to the standard 
contingency allowances of approximately 
10 percent, gave managers an extra degree 
of flexibility. However, certain elements 
of the project, particularly consultant fees, 
were significantly higher than budgeted, 
and consumed most of the unspent 
inflation allowance. In some cases, this 
financial flexibility was used to enhance 


————————— 


Canadian Security Intelligence Service 


the functionality of the building, such as 
the addition of the unfinished fifth floor 
south and the addition of a conference 
centre. 


8.81 We are not implying that there 
was no budgetary discipline. For the most 
part, the project budget was strictly 
controlled. Attempts to adhere to 
functional requirements were successful 
and original budget estimates for the 
different elements were respected. 
Additions or changes made were generally 
based on sound judgment and resulted in 
good value. 


8.82 There are always pressures to 
enhance the quality of systems and 
finishes on projects. We noted that, in 
many cases, design decisions were based 
on their relative value and how they 
would affect the budget. The rigorous 
analysis of all space requirements is a 
good example. However, the reduced 
pressure on the budget also led to a 
“build-to-budget ” philosophy. For 
example, the architects favoured granite 
panels for the building’s exterior cladding. 
The project team, concerned about the 
budget, also examined the costs for a brick 
and precast concrete cladding. When it 
was determined that granite could be used 
within the budget, it was decided to 
proceed without additional study of less 
expensive alternatives. Without 
commenting on the aesthetics of either 
choice, this was a potential opportunity 
for a cost saving to the taxpayer that was 
not fully explored. 


8.83 The Treasury Board Secretariat 
should clarify what the project approval 
means, stress lowest cost and consider 
incentives to encourage underspending 
budgets. Departments should be 
encouraged to identify and implement 
cost-saving opportunities in their 
approved projects. 
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We consider this 
project to have been 
well managed. 


The building is located on 
an isolated site. 


Conclusions 


A well-managed project overall with 
useful lessons learned 


8.84 Overall, we consider this project 
to have been well managed, 
notwithstanding certain negative findings, 
especially when compared with other 
projects we have examined. For example, 
we noted that many of the problems we 
have cited in our previous reports did not 
occur in this project, and there are some 
useful lessons for other government 
projects to learn from and consider 
applying where appropriate. Our 
interviews with the key project staff 
identified some factors that contributed to 
the project’s success. 


e The project team. The project team 
was the key element to the success of this 
project. The team, which was made up of 
the architects and members of CSIS and 
Public Works, had a good composition of 
skills and experience and was dedicated to 
the project. 


e The project leader. The project 
leader was responsible for only the 
national headquarters project, had the 
proper combination of skills and 
experience and had support from senior 
management. 


e Wide consultation. The final users 
of the space were involved in the planning 
and design phases, participated in 
developing the specifications, signed off 
on the design, and participated in the 
inspection of the construction in technical 
areas. The building operating team was 
also brought in at an early stage and was 
consulted on design issues. This extensive 
consultation probably accounts for the 
high level of satisfaction with the final 
product. 


e Adequate planning time. There was 
adequate, but not excessive, planning 
time. According to the project manager, 
the initial planning time paid dividends 
during the construction. 


e Freezing the design. In most cases, 
changes to the design were cut off by the 
project team at an early stage. There is 
often a temptation to continue to refine 
the plans toward achieving perfection. 
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Organizational and other changes in the 
future will be accommodated by the 
flexibility designed into the building. 


¢ Thorough design review. An aspect 
of the planning and design process that 
was unique to this project was the role that 
security considerations played. When a 
space requirement was specified, the 
security requirements for that space were 
reviewed. People were forced to think 
about how the space would be used and 
how it related to other functions. We 
believe that this extra level of review 
resulted in improved functionality and 
efficiency of the building. 


e Complete tender documents. 
Tenders were not called until the tender 
documents were complete. An incomplete 
set of documents at the time of tendering 
usually leads to problems, resulting in 
extra charges or claims. 


e Two-stage tendering process. The 
need for high security on this project had a 
positive effect on the tendering process. 
CSIS invoked Section 6.c of the 
government contracting regulations to 
control who had access to the tender 
documents. Although the need for this 
action was obvious, it had the added 
benefit of allowing a two-stage tendering 
process. Expressions of interest were 
called for, interested companies were 
screened, and five companies were 
selected by lottery to bid. 


e Early error detection. Details are 
important to an intelligence organization 
and can mean the difference between 
success or failure of an operation. This 
penchant for details in CSIS was 
successfully applied to the review of the 
specifications, drawings and the final 
product. Mistakes were often caught 
before they became expensive. 


¢ Quality assurance. At an early 
stage, a commissioning manager, 
reporting to the project leader, was made 
part of the project team. The manager was 
responsible for all aspects of 
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commissioning, quality control and 
reporting. Quality assurance requirements 
were made part of the contract 
specifications. As a result, the project 
leader had verification that work was 
being completed in compliance with the 
technical specifications. 


¢ Schedule. The building was 
completed in advance of schedule. Two of 
the contributing factors were that: 


— approval was obtained to start 
construction of Phase II 
foundations five months before 
the construction freeze expired. 
This allowed underground work 
to proceed while ground 
conditions were favourable; and 


- when it became apparent that the 
completion of drawings for the 
main construction contract would 
be delayed, extra work was added 
to the foundation contract in the 
form of a change order. Failure to 
act could have resulted in a work 
stoppage on site and a loss of 
momentum. 


User’s leadership role a key success 
factor 


8.85 We believe that the key to the 
success of this project was that the user 
(CSIS) acknowledged accountability and 
acted appropriately. From the start, CSIS 
made it clear to all parties involved that 
this project was its responsibility and 
would meet CSIS needs. CSIS controlled 
the project budget and authorized all 
spending. In addition to establishing a 
dedicated and qualified project team, 
CSIS engaged experts to supplement its 
own expertise. It also reviewed and 
applied lessons learned from the 
management of other projects and from 
our past audit reports. 


The Treasury Board Secretariat’s 
comments: The Treasury Board 
Secretariat concurs with the conclusion in 
this chapter that this was a well-managed 


We believe that the key 
to the success of this 


project was that the 
user (CSIS) 
acknowledged 
accountability and 
acted appropriately. 
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project overall, with a strong project said, we note that the approach followed 
leader and team, in which leadership by in this project contributed significantly to 
the building user was a key success factor. the strict control of the project budget 

We concur with the recommendation for and, ultimately, to the success of the 
greater emphasis on cost savings within project itself. 


the overall approved budget. This being 
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% About the Audit 


Objectives 


The objectives of the audit were to determine whether the constructed national headquarters facility would 
meet the CSIS-stated objectives and the Treasury Board approvals, and whether the project was implemented 
with due regard to economy and efficiency. 


Criteria 


Our audit criteria were derived from our guide for auditing capital asset projects, as well as the appropriate 
Treasury Board policies and guidelines. 


Scope 


The audit examined all the major stages of this major Crown project. Specifically, we reviewed the needs 
definition, the options analysis, the project definition, the design and review process, the contracting process, 
change orders, project management, environmental assessment, Commissioning and post-project evaluation. 


Our audit commenced in November 1995 and was completed in March 1996. Given the size and complexity 
of this project and the limited time available, we did not audit detailed financial records. The post-project 
evaluation, a requirement for a major Crown project, was in progress but not completed by CSIS at the time 
of the audit. 


The audit did not address the CSIS mandate. However, in acquiring an understanding of the requirements for 
the facility, we confirmed that they were based on the existing mandate and were appropriate. 


Approach 


Audit evidence was collected through extensive interviews with the building project staff, and with CSIS staff 
as users of the building. We reviewed planning documents, submissions to the Treasury Board, project briefs, 
minutes of the Senior Project Advisory Committee meetings and project management meetings, 
correspondence, contract documents and annual reports. We inspected the building, from the roof to the 
basement, including the office space, special purpose space and building services space. 


We received a high level of co-operation from the project management team, which included people from 
CSIS and Public Works and the architects. Members of the general CSIS staff were also very helpful. The 
level of co-operation is particularly noteworthy given the security considerations relative to CSIS operations 
and the facility itself. 


Audit Team 


Edward Wood 
Darlene Mulligan 
Joe Reperto 


For information, please contact Reno Cyr, the responsible auditor. 
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Keeping Parliament Informed 


Need to develop indicators to monitor performance 


New Ways of Doing Business 


Cost recovery, cost avoidance and cost reduction 
Industry pays too little for services with little “public good” 
Cost avoidance and cost recovery opportunities introduce new risks 


About the Audit 


Exhibits 

9.4 Programs and Animal Health and Plant Protection Lines of Business 
within Agriculture and Agri-Food Canada 

92 Animal and Plant Health 

a3 Program Expenditures for 1994-95 

9.4 Gypsy Moth Alert 

2:5 Infested Apples 

9.6 A Historical Comparison of Resources Allocated to Brucellosis Testing 

ee | Further Efficiencies Possible 

es More Effective Use of Available Resources 

9.9 Bovine Spongiform Encephalopathy (“Mad Cow Disease”) 

9.10 The Pine Shoot Beetle 

9.11 Examples of Information for Parliament on Status of Quarantinable Disease 
or Pest Outbreaks 

2.12 Who Should Pay? The Case of Bluetongue 

B13 Examples of Public And Private Good 

9.14 Overall Cost-Recovery Rate 
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Agriculture and Agri-Food 
Canada 


{gaits calle Animal and Plant Health: 
ssistant Auditor General: Don Young ; 
Responsible Auditor: Douglas Timmins Inspection and Regulation 


Main Points 


91 Canada’s relative freedom from animal diseases and plant pests and its high health standards have 
contributed significantly to our nation’s ability to export annually over $17 billion of agriculture and forestry 
products. International recognition of the health status of our domestic plant and animal resources is necessary to 
maintain access to export markets. 


9.2 The changing global agricultural economy has increased the pressure on the Department to allow 
additional imports, increasing risk to Canada. The Department has responded by assessing the risk of imports and 
by reacting quickly to threats identified. At the same time, the Department has made extensive efforts to realign 
its business approach to health protection, while also responding to significant resource reductions. 


9.3 The Department has developed a risk assessment process and, as a result, is recognized internationally as 
a leader in this area. The process provides the Department with a scientific basis to assist in making risk 
management decisions. Domestic health threats, regulated diseases, weeds and toxic substances have not been and 
will not be assessed in the foreseeable future, using this analytical tool, unless the emphasis shifts away from 
import trade-related activities. Consequently, the Department lacks sufficient reliable information to rank threats 
based on greatest risk, set priorities on a program-wide basis or demonstrate that it has made the best use of its 
available resources. 


9.4 Program objectives have been expressed in measurable terms. However, management has developed few 
performance indicators or other summary information to allow parliamentarians to understand and assess planned 
and actual performance and departmental action in response to serious outbreaks of diseases and pests. 


opel Over 1,100 managers, inspectors and laboratory staff deliver the Animal and Plant Health Program, at a 
cost of more than $100 million a year. While the Department has acted on a number of opportunities for cost 
reduction and cost avoidance, it continues to operate the Program and to plan future activities without 
rationalizing the need for the existing level of resources. 


9.6 Cost recovery has been implemented for some services previously provided without charge and will be 
expanded to others, to offset significant reductions in program funding. Considering the difficulties associated 
with maintaining industry competitiveness and international market access, with identifying beneficiaries and with 
developing a flexible and consultative approach to cost recovery, the Department has made progress over the short 
term. Our main concern is that the amounts currently being recovered fall at least 60 percent short of the full costs 
of delivering the service — service of which industry is the primary beneficiary. 
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Introduction 


D7 Agriculture and Agri-Food 
Canada administers three Acts of 
Parliament related to animal and plant 
health — the Health of Animals Act, the 
Plant Protection Act and the Seeds Act . 
These Acts authorize the Minister of 
Agriculture and Agri-Food to control a 
large number of plant pests and diseases 
and toxic substances that affect animals. 
The Animal and Plant Health Program 
involves two “lines of business”: animal 


SS ————— 
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health and plant protection (see 
Exhibit 9.1). For each of these lines of 
business the Department carries out the 
following types of activities: 


* import activities — to prevent the 
introduction into Canada of 
undesirable/quarantinable diseases, 
insects, weeds and other dangerous 
entities of plant and animal origin; 


e surveillance activities — to identify, 


control or eradicate undesirable/quarantin- 
able diseases, insects, weeds and other 


—_———_—_—_—_—_—_—————— 


Exhibit 9.1 
AGRICULTURE AND : 
Department AGRI-FOOD CANADA Programs and Animal Health 
and Plant Protection Lines of 
Business within Agriculture 
and Agri-Food Canada 
Branch Food Production and 
Inspection Branch 
Program area not audited LZ 
| Program area audited 
Agri-Food Animal and Plant 
Programs* Safety and Health 


Standards 


(Previously Audited) 


! 


Lines of Business 


Program Activities 


Animal Health 


Plant Protection 


ae 


® Import Control 

® Surveillance 

@ Domestic Control 
@ Export Control 


* These Programs are set out as sub-activities of the Inspection and Regulation Activity 


in the Department’s Estimates Part III. 
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dangerous entities of plant and animal 
origin that gain entry into Canada; 


e domestic activities — to prevent, 
control or eradicate identified domestic 
animal diseases and plant pests of 
significance to human health and the 
economy; and 


e export activities — to provide 
inspection and health certification 
required for agri-food products being 
exported or traded interprovincially and 
internationally. 


9.8 Our audit focussed on assessing 
the regulatory and inspection aspects of 
the Department’s Animal and Plant Health 
Program. Other aspects of the food 
production process (see Exhibit 9.2) were 
not part of our detailed examination. 
Details about the objective and scope of 
our audit are presented at the end of the 
chapter. 


9.9 In the 6 March 1996 Budget, the 
government announced its intention to 
introduce legislation to create a single 
Food Inspection Agency that will include 
the activities of the Animal and Plant 
Health Program examined in this audit. 
This decision does not affect the nature, 
significance or relevance of the 
observations and recommendations 
contained in this chapter. 


A Period of Program Realignment 


9.10 Our last audit of the Animal and 
Plant Health Program was in 1988. At that 
time, we were concerned that the existing 
Plant Quarantine and Animal Disease and 
Protection Acts and Regulations were 
outdated and required amendments to 
provide needed sanctions and authorities. 
In 1990, the passage of the Plant 
Protection Act and Health of Animals Act 
provided a new program framework by 
permitting qualified persons other than 
public servants of Agriculture and 


Agri-Food Canada to carry out the duties 
under the legislation, and provided for a 
departmental cost-recovery mechanism. 


9.11 During 1991 and 1992, the 
Department came under pressure as 
significant costs and problems were 
encountered in the control of 
tuberculosis-infected elk in northwestern 
Canada and a viral disease in potatoes in 
Atlantic Canada. In addition, international 
trade issues, combined with increasing 
demands of producers, industry 
associations, communities, the provinces 
and agricultural diversification, rapidly 
escalated the need for a change in the 
approach to disease and pest control in 
Canada. 


9.12 The period from 1992 to 1996 
has been very busy for those involved in 
the Department’s Animal and Plant Health 
Program. Not only did they have to 
continue carrying out ongoing activities, 
respond to significant resource reductions 
and manage disease outbreaks, but they 
were also involved in an extensive effort 
to realign the entire federal approach to 
animal and plant health protection that 
had served agriculture for over 100 years. 


9.13 In 1992, the Food Production and 
Inspection Branch conducted a regulatory 
review of the Health of Animals Act, the 
Plant Protection Act and the Seeds Act 
with a view to improving industry 
competitiveness, using government 
resources more efficiently and effectively, 
reducing costs and improving service to 
industry and consumers, and ensuring an 
appropriate relationship between the costs 
and benefits of the regulatory regime. 


9.14 In 1993, the Branch issued policy 
direction through a document called the 
Business Alignment Plan, intended to 
change program delivery to meet the 
following four objectives: 
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eae Program area audited 


Vv. 
Imports | Humane Treatment 
of Animals 
= 
VV 
Crop 
Production Vv 
4a ? Export/ 
' Domestic Sales 
' 
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} Animal ; 
: Production : 
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Safeguard Human, 
Animal & Plant Health 


Enhance 
Marketability 


! i 


Control / Prevent / Eradicate / Diseases / 


Pests / Toxic Substances 


i 


PROGRAM ACTIVITIES 
@ Import Control 
®@ Surveillance 
@ Domestic Control 
® Export Control 


I 


PROGRAM SUB-ACTIVITIES 
Inspection 
Testing/Analysis 
Identification/Investigation 


Surveillance 


C) 
® 

@ 

@ 

® Certification/Permits/Licences 
® Quarantine 

@ Regulation/Standards 

@ Disposal/Compensation 

e 


Technology Development and 
Transfer 


Agriculture and Agri-Food Canada 


Exhibit 9.2 
Animal and Plant Health 


Agricultural Production 
Process and Program 
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Canada’s relative 
freedom from serious 
pests and diseases 
contributes to the 
marketability of its 
agricultural and 
forestry products. 


Exhibit 9.3 


Program Expenditures for 
1994-95 


Source: Food Production and 
Inspection Branch’s 1995 Business 
Alignment Plan 


e to move toward a market-driven 
system that uses market forces to 
determine the value and need for a service 
by industry; 


e to redefine the roles of governments 
and the private sector in the delivery of its 
regulatory programs and services; 


e to have the beneficiary assume the 
responsibility for paying for the cost of the 
service; and 


e to establish a mechanism that allows 
adaptation to future changes. 


9.15 The Department believes that 
developing a new “partnership culture” 
involving other federal agencies, 
provincial counterparts and industry is at 
least as important as having a businesslike 
approach to delivering service. 


Understanding the Role of the 
Department in Animal and Plant 
Health 


9.16 Dealing with threats to human, 
animal and plant health and facilitating 
trade is a large part of the work of the 
Department’s Food Production and 
Inspection Branch. One aspect of this 
work deals with threats to Canada’s 
agriculture and forestry industries and 
requires programs and activities that will 
protect the health of this resource base. 


Canada’s relative freedom from serious 
diseases and pests contributes to the 
marketability of its agricultural and 
forestry products. For 1995, the total value 
of Canada’s agricultural and forestry 
products, excluding lumber and sawmill 
products, exceeds $50 billion. Exports of 
primary agricultural and forestry products, 
excluding lumber and sawmill products, 
total approximately $17 billion a year. 


O17 The budget for the Branch’s 
Animal and Plant Health Program is 
approximately $107.4 million a year (see 
Exhibit 9.3), based on 1994-95 operating 
expenditures of $74.4 million for animal 
health and $33.0 million for plant 
protection. 


9.18 Canadian industries affected by 
the Animal and Plant Health Program are 
varied, diverse and distributed throughout 
the nation. The primary industries are 
raising animals and growing plants for 
food and other purposes. Others involve 
purebred stock, horses for racing and 
pleasure, pets, zoo animals, ranched game 
species, wildlife in recreational areas and 
wildlife habitat. 


9.19 Canada’s agricultural and forestry 
industries continually look for animal and 
plant products from other countries to 
enhance the productive capacity of 
existing resources or to develop new 


: : : | 
($ millions) 
| Animal Health — | Plant Protection 
Import 16.9 Theil 
Export LOR) 6.7 
Domestic Control and Surveillance ODA rs 
Other* 15)! Hall 
oS 
TOTAL COSTS 74.4 33.0 


* Other costs include equipment, minor construction, accommodation, and corporate overhead. 


These costs are non—recoverable from user fees. 
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products and markets for consumption 
domestically and internationally. 
Importation is therefore conducted in an 
environment where health risks have to be 
assessed and managed. 


9.20 Canada has laws and regulations 
to prohibit or restrict the entry of animals 
or plants and their products from foreign 
countries. The Department uses several 
tools in Canada and in the country of 
origin to reduce the risk of importing 
health threats. In almost all cases, import 
permits are required for each shipment to 
control the movement of foreign animals 
or plants that may carry a disease or pest 
that poses a threat to human health or the 
economy. Further, the Department usually 
requires diagnostic tests for specific 
conditions, certification of the health of 
the commodity, and attestation by the 
foreign government that the procedures 
and requirements as set out in the import 
permit have been met. In addition, 
retesting, reinspection, and quarantine 
upon entry into Canada may be required 
to provide additional assurances. 


9.21 As a further control over imports, 
foreign animals and plants are subject to 
possible inspection at land, sea, air and 
mail ports of entry into Canada. Millions 
of tourists and approximately 250,000 
immigrants and their effects arrive 
annually. Customs officials detain goods 
and notify inspectors from Agriculture and 
Agri-Food Canada as animal and plant 
commodities arrive at the border with 
passengers or cargo. 


O22 The Department is charged with 
eradicating or controlling undesirable 
foreign or indigenous animal diseases or 
plant infestations that have serious 
economic implications for Canada, and 
with protecting the Canadian public 
against life- and health-threatening 
diseases from animals within Canada. 
Canadian law requires that certain 
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diseases and pests identified in Canada be 
reported to the Department. To fulfill its 
mandate, the Department conducts 
investigations, monitors and identifies 
diseases and pests, controls outbreaks (i.e. 
testing, isolation, treatment, destruction 
and disposal) and restricts the movement 
of animals and plants in specified 
circumstances. Several laboratories 
support these activities by providing 
diagnostic services, expert advice and 
technology development and transfer. 


9.23 The Department’s involvement in 
export operations is driven by 
international conventions that, for most 
plant and animal exports, require Canada 
to certify that the commodities meet the 
import requirements of the destination 
country. These requirements are 
negotiated between Canada and the 
importing country, and may involve 
inspection, testing and pre-shipment 
quarantine in Canada. Export market 
access is also dependent on maintaining 
international acceptance of Canada’s 
domestic control and surveillance 
activities. 


9.24 Because Canada exports a much 
greater volume (and value) than it 
imports, export certification requires 
significantly more resources. For many 
years, Canada has used accredited 
veterinarians to certify the health of 
animals, with the Department “endorsing” 
their work through monitoring. The use of 
accreditation is being expanded to cover 
the export of certain low-risk plant 
products to the United States and the 
export of seeds. Canada remains 
responsible for monitoring the accredited 
professionals and for ensuring that the 
quality of the service they provide 
continues to meet the standards required 
by our trading partners. 


9.25 Unlike imports, exports do not 
threaten the health of Canadians or 
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Canada is recognized 
internationally as a 
leader in the 
development of 
scientific risk 
assessment. 


Canada’s animal and plant resources. The 
direct beneficiary of these services is the 
exporter, whose access to markets is 
protected. The only significant risk for 
Canada is an economic one, in that 
exporting an unhealthy animal or plant 
that does not meet the import 
requirements of the recipient could 
threaten future exports to that country or 
other countries. This could, in turn, 
damage the Canadian economy and 
accordingly affect a much broader base of 
Canadians. 


Observations and 
Recommendations 


Program Design and Delivery 


An internationally recognized risk 
assessment process has been developed 


9.26 The Department has defined risk 
assessment as a process that 
systematically identifies threats that affect 
the health of humans, animals or plants, 
and the probability of negative 
consequences. It has further noted that 
systematic risk assessment provides the 


scientific basis for managing those threats. 


These interpretations are consistent with 
those of the international animal and plant 
health organizations to which Canada 
belongs. 


9.27 To facilitate the risk assessment 
process and in response to our 
recommendation in 1988, the Department 
has developed the Animal and Plant 
Health Risk Assessment Network 
(APHRAN). This risk assessment process 
allows the Department to answer these 
questions: 


e What is the threat? 


¢ What is the probability of entry, 
transmission and spread? 
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e What are the biological, economic 
and environmental impacts? 


9.28 In developing this process the 
Department now has a scientific, 
systematic and rigorous basis for 
completing risk assessments. These risk 
assessments are based on scientific and 
economic information and the process can 
be applied to any type of commodity, 
disease or pest. Further, it meets the 
standards of international treaties and 
conventions. As a result of this process, 
Canada is recognized internationally as a 
leader in the development of scientific 
risk assessment. 


9.29 The Department has dedicated 
resources to carry out scientific risk 
assessments of specific commodities, 
diseases or pests, using APHRAN. For 
animal health more than 50 risk 
assessments have been completed. These 
cover a wide range of commodities from 
several countries and include exotic 
commodities such as embryos and ostrich 
eggs, as well as more common 
commodities such as cattle and swine. 
Close to 300 risk assessments have been 
completed for plants. These include 
commodities such as bark (on imported 
logs), potatoes and apples, and pests such 
as hemlock woolly adelgid (a pest of 
hemlock) and corn cyst nematode (a pest 
of corn). The Department now has 
developed a common and consistent 
approach to implement the risk assessment 
process. 


Information is critical to scientific risk 
assessment and risk management 


9.30 The quality and nature of the 


information used has a direct impact on 
the value of the completed risk 
assessment. Disease- and pest-specific risk 
assessments rely mainly on scientific and 
economic information about the diseases 
or pests. 
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9.31 We found that the scientific risk 
assessments completed by the Department 
provide valuable scientific information on 
the threats associated with specific 
diseases or pests. However, in 11 cases we 
examined, the assessment of the economic 
impact of the threat could be strengthened, 
improving the overall value of the 
assessment. More specifically, there was 
limited information on who would benefit, 
who would be at risk, the volume and 
value of the commodity at risk and other 
commodities that might potentially be at 
risk, and possible trade restrictions and the 
value of such restrictions. Sources of 
economic information are not well 
developed, making it difficult to assess the 
economic impact of a threat. 


9.32 Commodity-specific risk 
assessments rely on information about the 
country wishing to export the commodity 
to Canada as well as information on 
specific diseases or pests. Of particular 
importance is information on new and 
emerging diseases and pests and on the 
disease and pest management practices of 
the exporting country. Therefore, we 
expected that the Department would 
integrate scientific and economic 
information on existing and emerging 
threats and on disease and pest 
management practices of an exporting 
country in the risk assessment process and 
in managing risk. The Department 


ee 
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continues to develop valuable 
international contacts and to identify new 
sources of information. These sources are 
particularly valuable in providing 
intelligence on existing and emerging 
threats internationally, and on the disease 
and pest management practices of other 
countries. However, this intelligence 
comes from a number of sources and some 
of it is gathered informally. It is also 
integrated into the risk assessment process 
in an informal way. In Plant Protection, 
this is further complicated by the fact that 
international reporting of pest outbreaks is 
not as fully developed as it is for Animal 
Health. As an international leader in risk 
assessment, there is a significant role for 
the Department in facilitating the 
development of early warning systems and 
reporting requirements by all nations 
participating in international plant health 
organizations. 


9.33 Exhibit 9.4 illustrates that, as a 
result of the lack of an international 
reporting requirement, an emerging threat 
was not identified as early as possible, 
which resulted in a delayed response by 
the Department. As noted in Exhibit 9.5, 
the Department exposed itself to the risk 
of importing an exotic pest associated 
with an imported commodity that was 
poorly managed and therefore posed a 
greater risk of carrying pests. 


Germany was experiencing a gypsy moth outbreak at the time that military equipment was 
being returned to Canada due to the closing of the Canadian Forces base there. Once the 
Department was aware of the outbreak it recognized the potential threat in bringing back 
equipment from the base. After some equipment had already been returned to Canada, the 
Department was informed by a contact in the United States that a shipment of American 
military equipment returning from Germany had been infested with gypsy moths. 

. Agriculture and Agri-Food Canada reacted quickly and, with the Department of National 
Defence, examined both the equipment already returned and that yet to arrive. In this case, 
the contacts in the United States proved very valuable in allowing Canada to respond to a 
threat. However, more formal international reporting requirements may have identified the 
potential threat before any equipment had been returned. 
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Gypsy Moth Alert 
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Gypsy Moth — male adult 
(see Exhibit 9.4) 


9.34 The Department should: 


e assess the economic impact of 
threats more comprehensively; 


e facilitate the development of early 
warning systems and international 
reporting requirements for pest 
outbreaks, by nations participating in 
international plant health 
organizations; and 


¢ be more rigorous in using 
information gathered on disease and 
pest management practices in other 
countries. 


Department’s response: Canada is 
recognized as a world leader in risk 
assessment. In 1994, the Department 
formally incorporated economic impact 
assessments of potential disease or pest 
threats as a component of the APHRAN 
risk assessment model. Notwithstanding 
the challenges posed by constantly 
changing international trade patterns and 
market conditions, the Department 
accepts that there is room to enhance the 


application of this relatively new tool as 
part of its overall risk analysis process. 


As a complement to existing international 
disease/pest information and intelligence 
gathering and monitoring practices, the 
Department will continue to advocate the 
need for more formalized transparent, 
co-operative international approaches. 


Several projects have already been 
initiated to address these concerns. They 
include: 


e the proposed development of an 
instrument to assess the competency of the 
regulatory sanitary infrastructure in other 
countries; 


e the formalization of an agreement in 
principle with several major trading 
partners to share risk assessments, 
inspection reports and audits undertaken 
of third countries; 


e the establishment of a 
geographic-based international trade 
team to build relationships with regulatory 
counterparts in assigned areas and to 
better network with agricultural trade and 
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commercial staff at Canadian embassies 
abroad; and 


¢ the establishment of a position to 
provide economic impact assessment to 
support the risk management process. 


Scientific risk assessments have focussed 


on import trade-related activities 


9°35 Given this internationally 
recognized process, we expected that the 
Department would rigorously apply it to 
all regulated diseases, pests, weeds and 
toxic substances. 


9.36 To date, the main focus of the 
assessments has been on specific import 
requests, reflecting the emphasis the 
Department is placing on trade-related 
activities to protect Canada’s health status 
and the urgency associated with import 
requests. The completed assessments for 
import requests have been considered in 
modifying program design and delivery. 
Most regulated pests have been subjected 
to scientific risk assessments, but no 
assessments have been completed on 
regulated diseases, weeds or toxic 
substances. Tuberculosis, brucellosis and 
rabies, which consume 90 percent of the 
animal health resources allocated to 
domestic activities ($22.4 million), have 
not undergone scientific risk assessments. 


———— eee 
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Unless the emphasis shifts from 
consideration of import health threats, 
domestic health threats as well as 
regulated diseases, weeds and toxic 
substances will not be assessed in the 
foreseeable future. 


937 While regulated-disease 
assessments rarely have the urgency of 
trade health-related assessments, the 
Department’s mandate implies that it 
needs to be able to demonstrate how it has 
dealt with threats while balancing its 
responsibilities for domestic health issues 
and trade obligations. As the demand 
increases for services to facilitate 
international trade, so does the importance 
of having a means to ensure that domestic 
health issues and trade obligations are 
balanced appropriately. 


9.38 Currently, diseases, pests and 
weeds are regulated while toxic 
substances are not. The Department has 
interpreted its role as proactive with 
respect to diseases and pests and reactive 
with respect to toxic substances. It does 
not actively encourage preventive 
measures and takes action only if advised 
that animals have been exposed to or 
contaminated by toxic materials. As a 
result, regulations have not been 
developed on tolerance levels and 


The Department accepted apples for importation based on a scientific risk assessment and 
followed its practice of permitting entry on a trial basis. Information supplied by the 
exporting country on the pest management practices, more specifically the processing and 


_ packaging practices, of the country of origin was considered in the scientific risk 


assessment. Nevertheless, upon arrival in Canada the apples were inspected and found to be 
infested, and required treatment. 


Earlier applications for import were denied because the country of origin did not provide 
sufficient information for Canada to adequately assess the pest management practices. After 
sufficient information was obtained, trial shipments of apples took place. When pests were 
intercepted during these trial shipments, Canada realized that modifications to the import 
conditions were needed. 

The arrival of infected apples in Canada demonstrates that reconsideration of the scientific 


basis for import approval may be required, once actual importation takes place. This case 
also shows that import approval needs to anticipate possible changes in the import 


i Serae and that stricter import conditions may be necessary in certain situations. 


Risk assessments 
have not been 
completed for 
regulated diseases, 
weeds or toxic 
substances. 


The Department has 
interpreted its role as 
proactive with respect 
to diseases and pests 
and reactive with 
respect to toxic 
substances. 


Exhibit 9.5 


Infested Apples 
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With increasing 
pressure to reduce 
resources, basing 
priorities on risk 


becomes important. 


standards for toxic substances that may 
affect animals or be transmitted from 
animals to humans. The Department’s 
interpretation of responsibilities set out in 
the Health of Animals Act for diseases 1s 
not the same as for toxic substances. 


9.39 The Department should: 


e complete scientific risk assessments 
for all regulated diseases and pests, as 
well as for other significant diseases, 
pests, weeds and toxic substances; and 


e ensure that it is meeting its 
legislative responsibilities in the priority 
it assigns to toxic substances that may 
affect animals or be transmitted from 
animals to humans. 


Department’s response: The Department 
accepts the recommendation. Progress 
against this recommendation will reflect 
an appropriate balance between the need 
to conduct these risk assessments and the 
Department’s commitments to other 
higher-risk areas such as incursions of 
diseases of major economic, health and 
safety significance (i.e. bovine spongiform 
encephalopathy and karnal bunt) and to 
international trade obligations. Closer 
integration between programs and the 
Animal and Plant Health Risk Assessment 
Network is planned to ensure that 
consensus agreement exists in establishing 
risk assessment priorities. 


The establishment of the Single Food 
Inspection Agency will provide a tighter 
integration of standards development 
between animal and human health risks. 
The Department will carry out a risk 
assessment to identify and prioritize 
known toxic substances. 


Program-wide priorities and resource 
allocation not based on scientific risk 
assessment 


9.40 We also expected that to manage 
risk, the results of scientific risk 
assessments would be incorporated with 


social, economic and political 
considerations and trade obligations in 
setting priorities and allocating resources 
at a program-wide level. Recently the 
Department completed two risk-based 
initiatives in which ratings of high, 
medium or low risk were assigned to 
generic import commodities, to assist in 
import decisions and to permit elimination 
of regulatory involvement in no-risk 
commodities. These initiatives reflect 
ongoing efforts by the Department to 
consider risk and these other 
considerations and obligations in setting 
priorities. A similar initiative for domestic 
activities has not been conducted. These 
initiatives need to be expanded to all 
activities so that priorities can be set and 
resources allocated among the various 
lines of business. 


9.41 An approach has been developed 
for ranking plant pests based on questions 
of biological significance covered in the 
APHRAN process. A similar approach for 
ranking animal diseases has not been 
developed. This approach for ranking 
plant pests could be used to categorize 
risks into high, medium and low for all 
domestic health threats of economic 
significance as well as regulated diseases 
and pests. This categorization would allow 
the Department to focus future scientific 
risk assessment efforts on the highest-risk 
areas and to incorporate the results in 
program-wide priority setting and 
allocating resources. Further, the level of 
federal involvement in medium- and 
low-risk threats could be reassessed. For 
low or no risk, federal involvement could 
be reduced or possibly discontinued. 


9.42 With increasing pressure to 
reduce resources, basing priorities on risk 
becomes more important. Yet it must be 
recognized that scientific risk assessment 
is a complex and time-consuming activity. 
When it is used for program-wide priority 


Report of the Auditor General of Canada — May 1996 


setting and allocating resources there is 
the added complexity of making decisions 
on risk involving diverse program 
activities (animal health and plant 
protection), each dealing with different 
commodities and each posing different 
degrees of risk to health or to international 
trade. 


9.43 Since our last audit of this 
Program in 1988, the responsibility for the 
day-to-day risk management has been 
delegated to the regions. This change has 
given the regions flexibility to adapt their 
operations to specific regional needs and 
circumstances. However, as the 
responsibility for program delivery 
becomes more decentralized, the concept 
of a national program may become 
threatened. Decisions on resources within 
the regions are complicated by the need to 
justify resource allocations among animal 
health, plant protection and the Branch’s » 
other lines of business (for example, meat 
hygiene). 


9.44 The plant protection line of 
business began developing its plans for the 
1996-97 fiscal year using a regionally 
focussed approach that incorporated 
consideration of risk based on the 
cumulative knowledge and experience of 
both the national and the regional plant 
protection managers. While the 
determination of risk was not as rigorous 
as formal risk assessments would be, the 
approach was a reasonable attempt to 
consider risk in allocating resources. 


9.45 For animal health, there has been 
an attempt to prioritize the various 
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components and activities of the animal 
health line of business using the practical 
experience of animal health staff. This has 
not significantly affected the allocation of 
resources, although the import inspections 
of low-risk animal products and 
by-products may be decreased or 
discontinued. Without the assurance 
provided by risk assessment, the current 
approach could result in resources being 
allocated to areas that do not pose the 
highest risk. 


9.46 Canada declared itself virtually 
free of brucellosis in 1985 and 
subsequently reduced testing, primarily 
based on the evidence of low incidence of 
this disease. The incidence of confirmed 
cases of brucellosis in Canada decreased 
from one in 1988-89 to zero in 1994-95 
and the number of tests conducted and 
staff assigned have been reduced (see 
Exhibit 9.6). The Department has recently 
established a centre of expertise to deal 
with this disease, even though the number 
of tests conducted and resources allocated 
for brucellosis have not been based on a 
scientific risk assessment. Testing is 
carried out at a level accepted by trading 
partners, but may exceed appropriate 
levels relative to other program risks. 


The current approach 
could result in 
resources being 
allocated to areas that 
do not pose the 
highest risk. 


9.47 The Department should make 
more effective use of scientific risk 
assessments, in conjunction with other 
considerations, in determining 
program-wide priorities and allocating 
resources to program areas where risks 
are greatest. 


Exhibit 9.6 
Number of Number of Number of A Historical Comparison of 
Year brucellosis tests reported incidents staff assigned Resources Allocated to 
= Brucellosis Testing 

1988-89 789,123 | 34 

— 

1994-95 428,796 0 24 Source: Agriculture and 
Agri-Food Canada 
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Department’s response: In addition to 
scientific risk assessments, the 
Department's priority setting and resource 
allocation practices must reflect other 
important factors such as multilateral 
trade agreement obligations, risk 
management and communication 
concepts, consumer confidence, societal 
values, industry interests, political 
realities, resource constraints and 
international credibility/fintegrity. The 
Department will explore opportunities to 
use risk assessments more effectively in 
determining program-wide priorities and 
allocating resources. 


Lack of reliable information to monitor 
program delivery against plans 


9.48 We expected that the Department 
would compare the actual work performed 
with the work that had been planned. We 
expected that the Department would 
know, as a minimum, whether desired 
inspection levels were being achieved, 
what had not been inspected but should 
have been, and what diseases or pests 
were found as a result of the inspections. 
This includes import, export, surveillance, 
and domestic activities. 


9.49 During 1995-96, the Department 
introduced the first phase of a new 
information system, the Resource 
Management System. It tracks the number 
of inspections performed and the 
resources used based on time standards, 
but does not track the volume of products 
inspected and not inspected. The new 
system is an improvement in that it 
provides uniform time standards for 
activities, making comparisons among 
regions possible. The Department has 
indicated that it also plans to use the new 
system to identify locations with efficient 
inspection practices that can be used to 
improve operations at other locations. 
However, there is a risk that gaps are not 
identified between what needs to be 
inspected and what actually is inspected. 


Since the Resource Management System 
does not track commodity shipment totals, 
it is not possible to determine whether 

50 percent or 10 percent of a specific 
commodity has been inspected. This 
information may be known by less formal 
means in the regions, but is not regularly 
available to or monitored by program 
managers at headquarters. 


9.50 Both the new system and the old 
system show considerable variation 
between the planned and actual number of 
inspections for any given region. We were 
advised that these differences were the 
result of unexpected fluctuations in the 
volume of products inspected. The regions 
need to have the flexibility to respond to 
unexpected volumes of shipments. But 
any increase in inspections of one 
commodity would be at the expense of the 
level of inspection planned for a different 
commodity. We found that the extent of 
such changes during the year by the 
regions, and the total products available 
for inspection, currently are not captured 
or summarized even though this 
information would be valuable for future 
years’ planning. 


951 The fact that some high-priority 
inspections were not being performed was 
included in a recent departmental 
evaluation report on plant protection. As 
currently designed, the Resource 
Management System will not identify 
those areas where work was not performed 
but should have been. Data on actual 
inspections compared with target levels 
would help managers correct program 
delivery problems sooner. 


9.52 The new system also does not 
track the results of inspections. This 
would be useful for costing purposes. 
Also, the number of “problems” identified 
during inspections would provide valuable 
information about the actual incidence of 
disease and pests experienced by each 
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region. This information is crucial to 
resource allocation. Changing patterns in 
disease or pest incidence could also be 
used in future years’ planning as a basis to 
reallocate resources among regions. 


D.59 Enhancements to the Resource 
Management System are planned for 
1996-97, but these will not address the 
lack of information on inspections 
performed. The system does track 
activities performed and thus provides 
information for billing of services 
performed by the Department. However, 
its usefulness as a management tool is 
limited by the lack of performance 
information and of variance reporting. 


9.54 The Department should ensure 
that it has reliable management 
information that allows a comparison of 
planned and actual program delivery, 
and explain how variances from plan 
will be addressed. 


Department’s response: One of the major 
undertakings of the Food Production and 
Inspection Branch in 1994 was the 
development of a sophisticated 
computer-based system that tracks the 
work of inspectors. 


The first phase of the Resource 
Management System (RMS) was 
implemented last year. The need for 
increased accountability was identified by 
the Department as a priority through an 
internal review. As a result, enhancements 
to the RMS planned for implementation in 
fiscal year 1996-97 will enable the 
Department to set out national standards 
based on inspection priorities that target 
resources for program delivery. Actual 
resource utilization will be measured 
against planned, allowing management 
analysis and decisions on reallocation of 
resources, program priorities, etc. 
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Opportunities for efficiencies need more 
examination 


Be In addition to having the 
necessary information to allocate 
resources properly and to compare 
planned activity levels to those delivered, 
the Department must be able to determine 
whether the program activities are being 
managed properly. We expected that there 
would be clear measures of program 
performance that could be used by 
program managers to improve the 
efficiency and effectiveness of program 
delivery. This information is necessary to 
answer questions such as: 


High-priority 
inspections were not 
being performed, 
according to a recent 
departmental 
evaluation report on 
plant protection. 


¢ How successful is a particular port of 
entry in preventing the introduction of 
diseases and pests? 


¢ Has the rate of compliance improved 
since the introduction of the revised 
regulations? 


¢ Do current shipments from a 
particular country contain fewer pests than 
one or two years ago? 


9.56 We found that information to 
answer these questions was not readily 
available. The Department does not make 
it a regular practice to collect and 
summarize the incidence of diseases or 
pests found during inspections. It does 
collect this type of information for 
inspections of passengers on international 
flights at major airports, but this is done 
months later, and represents only a small 
portion of the total number of inspections 
done during the year. Nevertheless, this 
type of result is a useful performance 
indicator for program delivery at both a 
regional and national level. 


9.57 There are other performance 
indicators currently being piloted by one 
of our major trading partners that have 
relevance to Canada. These indicators 
would attempt to measure the threat of 
diseases and/or pests gaining entry to 
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Opportunities for 
efficiencies may go 
undetected because of 
the lack of 
performance 
indicators. 


Canada and the success of the program in 
intercepting them. Some of these include: 


e Pest Threat Rate — an estimate of 
the proportion of units (cars, passengers, 
etc.) approaching a port of entry while 
carrying diseases or pests that could put 
Canada’s agricultural and forest sectors at 
risk; 

e Interception Gap — the difference 
between the estimated number of diseases 
and/or pests approaching a port of entry 
less the number actually intercepted; and 


e Targeting Efficiency Ratio —a 
measure of how much more effective 
risk-based inspections at ports of entry are 
than those carried out randomly. 


9.58 These performance indicators, or 
others with similar objectives, could 
provide the Department with additional 
information on whether the threat of pests 
gaining entry to Canada is increasing or 
decreasing; whether the Department is 
being more or less effective in 
intercepting pests before they gain entry; 
and how effective the Department is in 
focussing its limited inspection resources 
on high-risk shipments. 


9.59 The Department has made efforts 
to improve its efficiency and effectiveness 
in several areas. For example, it initiated 
the Enhanced Import Operations project in 
1993, with the objective of improving 
import compliance and service for 
shipments from the United States. One of 
the project’s goals is to increase the 
co-ordination between departmental 
inspectors and customs inspectors. The 
project also intends to accelerate the 
verification of documentation 
accompanying low-risk commodities, for 
importers who use the service frequently. 
Enhanced Import Operations plans to 
increase the effectiveness of import 
operations while providing a small cost 
saving through reduced overtime. While 
the project has not yet been implemented 


fully, it may eventually improve program 
monitoring by providing better 
information on the number of shipments 
inspected and not inspected. 


9.60 A second example of the 
Department’s efforts to increase 
effectiveness is the use of “blitz” 
inspections. At various times in the past 
three years the Department has conducted 
a “blitz” at import border points. A blitz is 
unannounced and involves a complete 
inspection of all agricultural shipments, or 
those of a specified commodity. As the 
use of blitzes continued, there was 
evidence of increased compliance with 
import regulations. This change was 
largely related to better compliance with 
documentation requirements. Since 
1994-95, responsibility for blitzes has 
shifted to the regions, with the result that 
blitzes may receive a different priority in 
each region. Since blitzes have a deterrent 
effect, headquarters needs to ensure that 
an appropriate number of blitzes are 
carried out in all regions. Blitzes could 
also be used to validate the choice of 
performance indicators and the 
effectiveness of targeted inspections. 


9.61 While some efficiencies in 
program delivery have been initiated, 
opportunities for others may go 
undetected because the lack of 
performance indicators inhibits their 
identification. For example, border 
inspection is an area where potential 
changes could increase the efficiency of 
program delivery. (See Exhibit 9.7 for 
details.) 


9.62 Also, by questioning how well 
the Program is delivered it is possible to 
improve effectiveness without incurring 
additional costs. Inspection at airports is 
an area where changes could increase 
effectiveness without additional cost, as 
shown in Exhibit 9.8. 
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9.63 The Department should further 
examine inspection and other activities 
with a view to identifying and 
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agencies in Canada and abroad have 
identified models that could be adopted 
for use. 


implementing additional efficiencies in 
program delivery. The Department 
should develop and use performance 
indicators that will aid in identifying 
opportunities to improve the efficiency 
and effectiveness of program delivery. 


Readiness to respond to unanticipated 
threats is uncertain 


9.64 Regardless of the number of 
inspections performed, or the stringency 
of the inspections, it is inevitable that a 
foreign animal disease or exotic pest will 
enter Canada. We expected that when 
faced with the incursion of a foreign 
animal disease or exotic pest, the Animal 
and Plant Health Program would be 
prepared to respond quickly and 
effectively. 


9.65 In the past, the Department has 
reacted quickly to a number of threats, 
including Asian gypsy moth, bovine 
spongiform encephalopathy (mad cow 


Department’s response: The Department 
is committed to pursuing creative and 
alternative means of achieving program 
goals. The Department will continue to 
build on improvements to program 
efficiency achieved through its 
three-pronged approach of cost avoidance, 
cost reduction and cost recovery for 
services; internal reviews; and the 199] 
Regulatory Review. 


We found that manuals 
were used in only one 
of the disease and 


pest incursions 
Program managers will endeavor to 


disease — see Exhibit 9.9), bluebe 
ensure that policies and procedures are mapeot atd-yesicular stomatitis tei responded to by the 
reviewed to provide for the most efficient fe 
eae ae, GAOT RE Ie ok oh action to control or eradicate the disease Department. 
integrity without incurring regulatory or pest before it could ap read. The ' 
liability. The program divisions are also emergency response actions taken in these 
committed to improving performance cases were not based on the emergency 
indicators, measurement tools and response manuals, but rather were 
corporate workplan development. managed through committees that were 
Communications with other regulatory formed to deal with each threat. Internal 

Exhibit 9.7 


Number of border inspection points. There are currently 215 ports of entry (border 
inspection points) where agricultural inspection services are offered, including airports, 
seaports, land border points and bonded warehouses. The majority of these ports of entry do 
not have agricultural inspectors on site at all times, but inspectors provide the necessary 
inspection services when called to the port. 


Further Efficiencies Possible 


As early as 1992, as part of the regulatory review, the Branch recognized that there were 
potential savings in limiting the number of ports. Internal studies have indicated that 65 
ports account for 97 percent of the volume of import traffic. Consultation with industry has 
shown an expectation for continued services at a similar number of ports. Although the 
Department has considered limiting the number of ports at which services are provided, it 
has not approached Cabinet for approval to make such a reduction. 


For the remaining 150 ports representing only 3 percent of the volume, inspections are 
infrequent and are carried out by inspectors who travel from other inspection offices to the 
port of entry. Many commodities are low-risk and are released without agricultural 
inspection. Infrequent inspection at a large number of ports of entry reduces the assurance 
that inspection procedures are carried out consistently. A reduction in the number of ports of 
entry would reduce this risk and would generate a small cost saving. If industry requests 
inspections at ports that do not warrant inspectors on site at all times, then the Department 
could recover any incremental costs associated with such inspections. 
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Exhibit 9.8 


reviews of the emergency response actions 
were completed subsequent to the 
elimination of the threat so that the 
Department could take advantage of the 
lessons learned from its response. 


9.66 To assist in departmental 
preparedness, the Department has 
developed manuals on emergency 
responses to foreign animal diseases and 
plant pests. These manuals provide 
guidance on who is responsible in an 
emergency and how they should respond. 
When the manuals were developed, 
training sessions were held across the 
country to educate staff on how to use 
them. Despite this effort, we found that 
the manuals were used in only one of the 
disease and pest incursions responded to 
by the Department. While several joint 
simulations involving headquarters and 
one or two regions have been completed, 
nation-wide simulations involving all 
regions at the same time have not been 


completed to ensure a uniform measure of 
readiness. Determining the need for 
regional simulations is at the discretion of 
each region. The result is that not all 
regions perform regular simulations. 
Similar to the emergency response review, 
simulations would highlight areas in the 
Department’s response that need 
improvement and would help ensure 
readiness for an actual emergency. We 
also found that the manuals were not 
updated to reflect the lessons learned from 
actual emergency response cases or from 
simulations conducted. A new initiative to 
update the manuals and make them more 
user-friendly is currently under way. The 
Department is also preparing documents 
for a number of foreign animal diseases to 
aid in industry’s understanding of 
procedures in the event of an incursion. 


9.67 The Department should 
conduct regular simulations and 
maintain updated procedures manuals 


Inspection of travellers at airports. During 1994-95, 11 million travellers arrived in 
Canadian airports on flights from other countries. On arrival, travellers are asked whether 
they are carrying any agricultural products and have visited or will visit farms. Based on their 
responses, a sample of travellers is selected for further examination. Currently, agricultural 
inspectors at airports in Canada inspect goods or travellers’ effects primarily when called 
upon by Revenue Canada customs inspectors. Flights arriving from origins having a higher 
risk of diseases and pests are intended to be sampled at a higher rate. However, no attempts 
have been made to measure the actual “threat rate” to Canada, and the Department does not 
estimate the number of high-risk goods that go undetected each year. In recent years the 
Department has used a limited number of “detector dogs” to help search for hidden animal 
and plant products in travellers’ luggage. 


More Effective Use of 
Available Resources 


There is an opportunity to increase the effectiveness of these activities. Procedures at airports 
are designed to intercept diseases and pests, but they could be designed to deter or prevent 
their introduction by being more visible. The awareness of travellers from high-risk countries 
that their effects are subject to a high level of scrutiny would be increased if these inspectors 
were more visible to the traveller and if there were greater use of “detector dogs”. Spot 
inspections could also be performed, and these inspections would help to determine whether 
customs officers had been able to identify all appropriate referrals for further inspection. 


Agricultural inspectors and detector dogs serve not only a functional role but also a deterrent 
role that could dissuade travellers from bringing restricted goods with them. The Department 
acknowledges that resources can be used more effectively and has indicated it will conduct, 
in conjunction with Revenue Canada, a study of inspections of travellers at airports and land 
border crossings. 
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to better ensure a readiness to respond 
to an incursion of a serious disease or 
pest. 


Department’s response: The Department 
appreciates the Auditor General’s 
recognition of its quick responses to past 
threats including Asian gypsy moth and 
bovine spongiform encephalopathy (mad 
cow disease). 


Regional simulations of emergency 
situations will continue to be held. 
Training initiatives and technical manuals 
are being revised in the key areas of 
cleaning/disinfection and 
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disposal/destruction. A contingency has 
been established to utilize emergency 
response opportunities and disease 
outbreaks external to Canada as training 
opportunities for critical elements of 
national and regional emergency 
situations. Also critical to emergency 
preparedness is the Department’s ability to 
detect/diagnose threats, train personnel in 
recognizing pest and animal diseases and 
appropriate response measures, provide 
scientific advice and develop improved 
detection technologies. Agriculture and 
Agri-Food Canada centres of expertise for 
plant quarantine pests and foreign animal 
diseases perform these functions. 


r 


period of the disease. 


along with seven other animals. 


cow disease” have been identified. 


In 1993 the Department responded to the first confirmed case of bovine spongiform 
encephalopathy in North America. The disease, also referred to as “mad cow disease” 
because of the effects it has on the behaviour of infected animals, is a slowly progressive 
fatal nervous disorder of cattle. It has claimed more than 160,000 cattle in the United 
Kingdom since it was first identified in 1986 and is currently threatening the national herd 
there. The circumstances that scientists believe led directly to the widespread nature of the 
disease in the United Kingdom do not exist in Canada. 


The Department was aware of the situation in the United Kingdom, and in 1990 stopped 
allowing the importation of live cattle from there. A surveillance program was also 
implemented in 1990, involving the tracing and routine examination of all cattle imported 
from the United Kingdom between 1982 and 1990, which covered the known incubation 


On 8 December 1993, the Department confirmed a case of bovine spongiform - 
_encephalopathy in central Alberta. The owner had notified departmental inspectors about 

unusual symptoms associated with bovine spongiform encephalopathy, as owners had been 

instructed to do. The cow had originally been imported from the United Kingdom in 1987, 


To protect both the Canadian cattle herd and related export markets valued at $1.7 billion, 
the entire herd in which the infected animal resided was destroyed, as well as all offspring of 
the infected animal. All other cattle imported to Canada from the United Kingdom since 
1982 that were still alive were also destroyed. In addition, all offspring born in Canada 
whose parents came from infected herds in the United Kingdom were destroyed. As a final 
precaution, the carcasses of all destroyed animals were incinerated. The Department’s 
actions allowed Canada to maintain access to world markets, and no further cases of “mad 


The approach to risk management and communication chosen by the Department to 
eliminate the threat to Canada posed by bovine spongiform encephalopathy was somewhat 
controversial at the time, primarily because it involved the destruction of 403 animals — 
more than just the direct herd mates of the infected animal, including animals with no 
clinical symptoms of disease. The Department took this action to maintain international 
markets and public confidence. As the current situation in the United Kingdom 
demonstrates, fear about the existence of this disease can result in significant reductions in 
beef consumption and loss of major export markets. 


Exhibit 9.9 


Bovine Spongiform 
Encephalopathy 
(“Mad Cow Disease”) 
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The Program is 
controlling the spread 
of most economically 
significant animal 
diseases and plant 
pests. 


Keeping Parliament Informed 


9.68 The Department’s Estimates Part 
II is one of the primary vehicles for 
parliamentarians to review expenditures. 
Part IIIs are intended to provide sufficient 
information to understand and assess 
planned and actual performance. We 
expected that to give parliamentarians the 
basic information they need, the Estimates 
Part III or other readily accessible 
documentation would: 


e clearly explain the objectives of the 
Program, preferably expressed in 
measurable terms; 


e demonstrate whether the “lines of 
business” were achieving the results 
intended, using key performance 
indicators and other relevant information; 
and 


e relate the “lines of business” and 
activities to the use of resources to meet 
the demands for service. 


Need to develop indicators to monitor 
performance 


9.69 In the Department’s 1995-96 
Estimates Part III, the part of its objective 
for the Inspection and Regulation 
programs that relates to the Animal and 
Plant Health Program is “to enhance the 
marketability of agricultural and food 
products by eliminating or controlling 
plant and animal diseases.” 


9.70 The Department has interpreted 
its Inspection and Regulation objective 
more specifically for the Animal and Plant 
Health Program as: 


e prevention of the entry into Canada 
of exotic diseases, insects, weeds and 
dangerous entities of plant or animal 
origin; 


e control or eradication of exotic 
diseases, insects, weeds and other 


dangerous entities of plant or animal 
origin that gain entry into Canada; 


¢ prevention, eradication and control 
of domestic animal or plant diseases and 
pests of economic or human health 
significance; and 


* provision of inspection and 
certification required for agri-food 
products being exported. 


9.71 These statements are a reasonable 
reflection of the intent of the Animal and 
Plant Health Program, and the outcomes 
could be measured. Measurement against 
specific expectations or targets would 
indicate whether the Program is achieving 
the results intended. 


9.72 The Animal and Plant Health 
Program is controlling the spread of most 
economically significant animal diseases 
and plant pests. As a result, we believe 
that most taxpayers would have difficulty 
identifying cases where disease or pest 
outbreaks have seriously threatened 
human health or significantly damaged 
our export trade. Yet we found that the 
Department reports only one performance 
indicator in support of results achieved — 
the number of foreign diseases or pests 
that have become established in Canada 
during the year. The 1995-96 Estimates 
Part III states that no reportable diseases 
have become “established in Canada” in 
the past three fiscal years. In actual fact, 
at least one pest has become “established” 
in Canada during this period (see 

Exhibit 9.10). 


9.73 There are two important 
limitations to the use of “established in 
Canada” as a performance indicator that 
need to be explained in the Part IIIs. First, 
“established in Canada” has different 
meanings for animal health and for plant 
protection. Second, the reliability of this 
indicator is affected by the number of 
surveys conducted by the Department. In 
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recent years only about 14 pest surveys 
have been conducted annually, while 
potential pests number in the hundreds. 


9.74 If clearly explained, substantiated 
and reported, this indicator would provide 
valuable insight into the Department’s 
ability to prevent the entry of foreign 
diseases and pests. As presently reported, 
however, it is inaccurate and possibly 
misleading. 


9,75 How the Department responds to 
disease or pest outbreaks may be relevant 
for Parliament and is critical for industry. 

It is not possible to prevent the occasional 
entry of foreign diseases or pests, no 
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matter how well designed and managed 


the Animal and Plant Health Program may 


be. International conventions require 
countries to immediately notify their 
trading partners whenever certain serious 
diseases (listed in the conventions) are 
identified. This obligation is designed to 
protect all trading nations by ensuring that 
all countries have timely information 
about potentially serious outbreaks of a 
disease or pest. Consequently, the 
Department’s tactical response to disease 
outbreaks is critical to maintaining and/or 
restoring the market access necessary for 
trade. Past responses represent what some 
in the Department believe have been its 
most significant contributions or results. 


The pine shoot beetle is a pest native to Eurasia that feeds on young pine tree growth. The 
first report of the pest occurred in the summer of 1992 in Ohio. Following further reports, the 
Department began surveys in Canada in 1993 and the beetle was found in seven Ontario 
counties. Quarantine zones have been imposed in 14 counties in the Toronto—Niagara 
horseshoe in order to prevent the spread of the beetle to northern pine forests and to other 
provinces. The real danger is that the beetle could become established in lumber stands where 
growth and yields could be seriously affected. The Department has not reported to Parliament 
that this pest is now established in Canada. 


The North American Plant Protection Organization defines “established” as “an introduced 
pest, present in a country or area, multiplying and expected to continue”. Within that 
definition are various categories of established pests. The pine shoot beetle meets the 
definition of a pest that is established within a limited area of distribution and is under active 
control. 


Common Pine Shoot Beetle — adult 


The Department’s 
tactical response to 
disease outbreaks is 
Critical to maintaining 
and/or restoring the 
market access 
necessary for trade. 


rr 


Exhibit 9.10 


The Pine Shoot Beetle 
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Exhibit 9.11 


Examples of Information for 
Parliament on Status of 
Quarantinable Disease or 
Pest Outbreaks 


9.76 Summary information in the 
Estimates Part III on results achieved in 
relation to objectives of prevention, 
control, eradication and export services 
would give Parliament the necessary 
performance information that is not 
currently provided there. This information 
could be a combination of quantitative 
information, such as the number of 
surveys conducted and the number of 
disease or pest outbreaks being controlled 
or eradicated, and qualitative information, 
such as the nature, significance and 
current status of outbreaks. Qualitative 
information, as described in Exhibit 9.11, 
might be provided only for significant 
outbreaks, whereas full details of all 
outbreaks, similar to the information 
already provided to trading partners, could 


be available to parliamentarians through 
other documents. 


9.77 The Department also needs to 
make available summary information on 
the volume, cost and resource 
requirements of the important program 
activities and sub-activities. The 
Department does provide some 
information, but it needs to be related 
more directly to the workload 
requirements of the activities involved. 
For example, in relation to the export 
services objective it would be useful to 
disclose (on a comparative-year basis) the 
number of export certificates issued and 
the value of the export products, the 
number of full-time staff and cost required 
to deliver services and the average cost 


Nature 
@ description of disease or pest 
@ animals or plants affected 
@ area infected — a 
®@ likelihood of spreading 
® human health impact, if any 


Market Affect ed : 
® commodities immediately affected 
@ annual value of commodity sales 


Potential Economic Significance 
@ other commodities potentially at risk 
® export markets potentially at risk _ 
_ @ additional costs to producers, if any 


Date First Reported 

Actions Planned/Taken 
® control or eradication measures 
© regulatory changes 
@ compensation, if applicable 


Date Resolved/Expected to be Resolved 
Anticipated Cost _ 


© markets immediately closed or restricted 


®@ one-time and ongoing costs to the Department 
@ costs expected to be recovered from industry 
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per certificate issued. Similar summary 
information could be provided for other 
major activities. Such information could 
be provided through a modification to, or 
in place of, the information currently 
shown. 


9.78 The Department has improved 
the information it provides to Parliament 
about the Program since our 1988 audit, 
and continues to do so. In the Preface to 
its 1995—96 Estimates Part III, the 
Department advises that “a process to 
completely revise the Operational 
Planning Framework is under way.” This 
Framework is used by the Department as 
the basis for planning, budgeting, 
expenditure management and performance 
monitoring. There is an opportunity for 
the Department to integrate our 
recommendations in this ongoing work. 


v2 The Department should give 
parliamentarians access to improved 
summary information by: 


e developing and reporting 
measurable performance indicators for 
all aspects of its objectives (prevention, 
control, eradication and export 
services); 


¢ specifically identifying disease or 
pest outbreaks of critical significance to 
human health or the economy that have 
occurred, and demonstrating how the 
Department has managed or reduced 
the threat to Canada; and 


¢ modifying information on use of 
resources to be more directly related to 
workload requirements. 


Department’s response: The Department 
agrees that its performance reporting to 
Parliament needs improvement and is 
committed to provide an updated and 
revised set of performance indicators to 
Parliament in the spring of 1997. These 
indicators will be identified in conjunction 
with the development of a new 
Operational Plan Framework based on 
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business lines. Planning and reporting 
against this new Framework will 
commence in 1997-98. As well, during the 
1995-96 consultations with industry, the 
Department is proposing standards that 
will describe the level of service that 
clients can expect in all functional areas. 


New Ways of Doing Business 


Cost recovery, cost avoidance and cost 
reduction 


9.80 The government’s recent program 
review identified the Department’s 
Animal and Plant Health Program as an 
area where the use of government 
resources could be reduced through 
various means. The program review 
followed a previous government-wide 
regulatory review, which had identified 
opportunities to reduce the regulatory 
burden on industry and, correspondingly, 
program costs to government. Based on 
these reviews, a total of approximately 
$23 million was removed from the 
Program’s funding reference levels in the 
period 1995—96 through 1997-98. This © 
represents approximately 21 percent of the 
cost of the Animal and Plant Health 
Program for 1994-95 — a significant 
reduction in resources. Accordingly, it is 
critical for the Department to manage this 
funding reduction well, in order to 
preserve the integrity of the Program and 
maintain an appropriate balance between 
health-related and trade-related objectives. 


9.81 The Department developed its 
Business Alignment Plan in response to 
these reduced funding levels. As described 
in its 1995-96 Estimates Part III, “The 
Department is aligning its program base in 
response to the market forces shaping the 
Canadian agri-food sector. Negotiations 
are under way with groups of stakeholders 
to select business arrangements that will 
permit the continued delivery of services 
critical to industry viability while enabling 
the Department to implement plans 
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The Business 
Alignment Plan was 
developed in response 
to significantly 
reduced funding 
levels. 


addressing the government’s fiscal 
restraint measures.” The planned 
realignment of the program base is 
proceeding along three main initiatives: 


e cost recovery — having the 
beneficiary of the service share in its cost; 


e cost avoidance — having someone 
other than the Department deliver and pay 
for the service (for example, 
privatization); and 


e cost reduction — reducing the cost 
of providing the service through greater 
efficiencies, eliminating unnecessary 
procedures or, in some cases, 
discontinuing the service. 


9.82 We expected that in 
implementing its Business Alignment Plan 
and in particular its cost-recovery 
initiatives, the Department would: 


e develop a long-term strategy for cost 
recovery based on identifying the 
beneficiary of each service provided and 
recovering costs proportionate to the share 
of private benefit derived; and 


e adopt a consultative and flexible 
approach with industry. 


Industry pays too little for services with 
little “public good” 


9.83 In developing its Business 
Alignment Plan, the Department looked at 
the portfolio of services it provided and 
segregated them into “basic” and 
“additional” services. “Basic” services 
were those that the Department considers 
mandatory and that support the integrity 
of the Animal and Plant Health Program. 
These services would be carried out even 
if no other requests, international 
requirements, etc. were present. Examples 
include activities such as explanations and 
interpretations of regulations, acts and 
policies; border point and import 
monitoring programs for health and 
safety; production inspection and/or 


monitoring at a frequency based on risk; 
and disease control. 


9.84 “Additional” services were 
defined by the Department as those 
provided in response to requests from 
industry, foreign country requirements, 
etc. Examples include activities such as: 
completion/endorsement of export health 
certificates; inspection services upon 
request; laboratory analysis in support of 
“additional” services; and so on. 


9.85 The Department also examined 
its activities from a slightly different 
perspective. It stated that a “beneficiary 
pays” philosophy should be adopted 
whenever there was an identifiable 
beneficiary of a service. A discussion 
paper prepared by the Department argued 
that “beneficiary participation in program 
funding will encourage a more realistic 
demand for services. Beneficiaries will 
determine which programs are worth 
supporting based on market forces (i.e. 
benefits vs. costs). This will facilitate the 
transition from government-run regulatory 
programs to industry-operated quality 
management programs for disease/pest 
control.” 


9.86 The Department noted that most 
of the activities conducted by the Animal 
and Plant Health Program had several 
beneficiaries, and that “such programs 
should therefore be operated as 
partnerships: business arrangements in 
which beneficiaries will be able to 
participate in program design, 
development and delivery, as well as 
funding.” While this may be quite 
appropriate, it can lead to complexities in 
identifying the beneficiary who should be 
asked to pay (see Exhibit 9.12). 


9.87 If a “beneficiary pays” 
philosophy were to be adopted, several 
principles would logically follow: 
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e the Department would give priority 
to funding activities that promote 
“national public good”; 


e the Department would encourage and 
facilitate beneficiary self-reliance where 
“private good” is concerned; and 


e the Department would apply the 
“beneficiary pays” principle equitably and 
consistently to all beneficiaries. 


9.88 While the notions of “beneficiary 
pays” and segregating “public” good from 
“private” good have their appeal, they are 
difficult to implement. Most services that 
are currently provided have elements of 
both private and public good. This point is 
illustrated in Exhibit 9.13. 


9.89 The Department was faced with a 
difficult task in determining a specific 
percentage of costs to be recovered. 
Rather than try to reach agreement on a 
specific allocation between private good 
and public good, it developed an 
alternative approach. For services that 
clearly had elements of both private good 
and public good, the Department argued 
that whatever the proportion of private 
good, it was at least 50 percent. This 
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approach had the advantage of avoiding 
arguments about whether the proportion of 
private good was, for example, 60 percent, 
70 percent or 95 percent. The solution of 
up to 50 percent as an initial cost-recovery 
target is appealing because it allowed the 
Department to move quickly in 
establishing its cost-recovery initiative. 
However, it may result in the public 
bearing a disproportionate share of the 
costs of activities that provide a limited 
contribution to the public good, 
particularly if the percentage of costs 
recovered is not revised in the future. 


9.90 To introduce a cost-recovery 
regime, the Department had to identify the 
costs of each of its activities. This was a 
complex task, because its financial and 
management control systems had not been 
designed with this in mind. Consequently, 
some judgment was required in allocating 
costs to services, and a considerable 
amount of analysis was performed to 
identify the resources used to provide each 
service. 


9.91 To the Department’s credit, 
reasonable costing principles were 
developed and the full costs of the Animal 


Most services 
currently provided 
have elements of both 
private and public 
good. 


[ 


market access for the cattle industry. 


However, bluetongue in sheep can be fatal. 


an issue. 


access to be maintained. 


regime. 


Canada is currently free of bluetongue. This health status is important in maintaining foreign 


Until recently, the cattle industry was required by Canada to have all imported animals from 
the United States tested for bluetongue. In cattle, bluetongue is a very mild disease not unlike 
a cold. Symptoms may not even be visible and the health of the animal is not at risk. 


With the advent of cost recovery, the question of who should pay for testing for bluetongue is 


On the one hand, it may be argued that those in the sheep industry are the beneficiaries of this 
service because it is their livelihood that is being protected. On the other hand, it may be 
argued that the cattle industry is the real beneficiary because testing allows foreign market 


In this case, the Department is treating the cattle industry as the beneficiary of the service. 


This simple example illustrates the complexity in establishing an equitable cost-recovery 


Exhibit 9.12 


Who Should Pay? 
The Case of Bluetongue 
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More of the 
cost-sharing burden 
should be borne by 
industry where public 
good is minimal. 


Exhibit 9.13 


and Plant Health Program were allocated 
to its two lines of business and its 
activities. The full costs were then 
subdivided into two categories, 
recoverable and non-recoverable costs. 
Non-recoverable costs included items such 
as corporate overhead, equipment and 
minor construction, accommodation and 
certain other costs. As a result, 
recoverable costs amount to 
approximately 80 percent of total costs. 


9.92 For those activities whose cost is 
being recovered, the current recovery 
percentage ranges from 30 percent to 50 
percent of “recoverable costs”. We believe 
that, to ensure equity and fairness, more of 
the cost-sharing burden should be borne 
by industry where public good is minimal. 


9.93 
recoverable costs will be much less than 
current rates would suggest. Exhibit 9.14 
indicates that even if all of the required 
savings from cost avoidance and cost 


The rate of recovery on overall 


Examples of Public And Private Good 


reduction are assumed to be 
“recoverable”, the overall rate of cost 
recovery will be quite low at 20 percent of 
total recoverable costs. The overall rate 
will be influenced by the future revisions 
to recovery percentages, progress in cost 
reduction and cost avoidance and possible 
legislative changes to facilitate the 
collection of fees. 


9.94 In response to industry concerns, 
we found that the Department made 
consultation with industry an important 
element of its Business Alignment Plan. 
The overall Plan was organized into three 
rounds, with implementation of the first 
round beginning in April 1995. For each 
round the Department prepared an initial 
information package on its proposals, and 
then discussed them with industry. This 
was followed by modifications based on 
the input received and follow-up 
discussions with industry to explain how 
its concerns had been addressed. 


ANIMAL AND PLANT 
HEALTH PROGRAM 
ACTIVITY 


- PUBLICGOOD 


PRIVATE GOOD 


Inspections of Animals or 
Plants with Import Permits 


The public benefits only indirectly in the 
sense that the activity is geared to protecting 
our already healthy plant and animal 
populations from threats due to import 
activities. 


The importer directly benefits by being able 
to bring into Canada an animal or plant free 
of disease. The importer is a risk maker. 
The risk may be of human health concern, 
or it may affect animals or plants of the 
same or of a different species. 


Export Health Certification 


There is a very small element of ‘“‘public” 
good. Canada as a whole benefits from 
access to world markets as a result of abiding 
by global trading conventions. A strong 
economy benefits all Canadians. 


The “good” is largely one of a private 
nature. The main beneficiary is the 
individual exporter. 


Border Inspections 


There is significant element of public good. 
The border inspections are intended to 
reduce the threat of entry to Canada of 
diseases or pests of a human health 
significance. 


There is only a small element of private 
good. It would be impractical to identify a 
primary beneficiary, since all Canadians 
would tend to benefit equally from these 
preventive measures. 
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35 One illustration of the 
Department’s flexibility was its response 
to industry’s concerns about the impact of 
user fees on its competitive position. 
Because such a significant portion of our 
trade is with the United States, there was 
pressure from industry to keep Canadian 
export-related user fees in line with those 
of the United States for similar services. 
While the Department wanted market 
forces to determine services to be 
provided, it did not want to drive industry 
out of markets by imposing expensive user 
fees. The Department has indicated that it 
is aware of the need to minimize the 
negative impact on industry 
competitiveness and international market 
access. This led it to establish cost 
recovery for export services at 25 percent 
to 30 percent of “recoverable” costs. 


9.96 Cost recovery for services 
previously provided without charge is 
currently being phased in over a 
three-year period to cover the full 
spectrum of program activities. We think 
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and develop a long-term plan for industry 
to pay a share of the costs of services that 
is commensurate with the degree of 
private benefit derived. 


9.97 The Department should 
determine the level of private good for 
each service provided and develop a 
strategy to recover an appropriate 
amount of revenue relating to the 
private benefit portion of the services. 


Department’s response: The Department 
accepts the recommendation and will 
proceed to develop a set of criteria that 
will guide the estimation of the level of 
private good associated with each 
functional area of service for each 
commodity group. These estimates of 
private good will be discussed with clients 
during the third round of cost recovery 
consultations, which are planned for 
1997-1998. Departmental progress 
against these estimated levels will be 
influenced by the evolution of foreign 
countries’ cost recovery policies and the 
importance of respecting the Canadian 
industry's ability to compete in these 


the Department needs to go even further markets. 
Exhibit 9.14 
Overall Cost-Recovery Rate 
ANIMAL HEALTH PLANT PROTECTION PROGRAM TOTAL 
1994-95 Recoverable Costs $ 59,065,700 $ 26,394,000 $ 85,459,700 
(Base Year information) 
Less: 
Savings from cost avoidance and/or cost re- $ 5,500,000 $ 2,400,000 $ 7,900,000 
duction 
(To be achieved over three years) 
Equals: 
Net Recoverable Costs (A) $ 53,565,700 $ 23,994,000 $ 77,559,700 
(Assumes savings from cost avoidance and 
cost reduction were “‘recoverable” — a con- 
servation assumption) 
Target Cost Recovery (B) $ 10,600,000 $ 4,600,000 $ 15,200,000 
(To be achieved at the end of three years) 
Overall Cost-Recovery Rate (B/A) 20% 19% 20% ed 


Source: Agriculture and Agri-Food Canada 


Report of the Auditor General of Canada — May 1996 


9-31 


Agriculture and Agri-Food Canada 


Accreditation is a 
reflection of the 
changing approach to 
delivering the Animal 
and Plant Health 
program. 


Cost avoidance and cost recovery 
opportunities introduce new risks 


9.98 The Business Alignment Plan is 
flexible in that “market forces” can 
influence the restructuring of the 
Department’s operations. The 
Department’s approach has been to work 
with industry to identify an optimal mix of 
cost recovery, cost avoidance (for 
example, accreditation and privatization) 
and cost reduction (for example, 
eliminating unnecessary services or 
performing existing services more 
efficiently and/or effectively) that would 
lead to meeting the financial targets set by 
the government. 


9.99 Accordingly, our expectations 
were that the Department would: 


e thoroughly re-examine all of its 
current services to ensure that they are 
being delivered as efficiently and 
effectively as possible; and 


e ensure that any changes to program 
delivery as a result of cost avoidance 
and/or cost reduction measures will not 
compromise the Animal and Plant Health 
Program objectives of safeguarding health 
and facilitating trade. 


9.100 The weaknesses in its systems for 
monitoring and reporting on program 
delivery limit the Department’s ability to 
identify areas where services could be 
delivered more efficiently or effectively. 
Until these weaknesses are rectified, there 
may be opportunities for cost avoidance 
and/or cost reduction that are not acted 
upon (see Exhibits 9.7 and 9.8). In spite of 
these shortcomings, the Department has 
identified opportunities for both cost 
avoidance and cost reduction. 


9.101 In the plant protection line of 
business, an example of cost reduction is 
the introduction of multi-year import 
permits for certain plants and plant 
products. This has reduced the number of 


import permits issued by plant protection 
in 1994-95 to approximately 6,000 from 
the 18,000 that were issued in 1992-93. 
An example of cost avoidance is the 
Department’s move toward the use of 
industry accreditation for the export of 
greenhouse plants. This will eliminate the 
need to issue approximately 10,000 export 
health certificates annually. 


9.102 Cost avoidance through the use 
of accreditation is not new to the 
Department. The inspection and testing of 
animals destined for export (primarily to 
the United States but also to Mexico) has 
been done for many years by private 
sector veterinarians who have been 
“accredited” by the Department. These 
accredited veterinarians are also 
responsible for completing the related 
export health certificates, which are 
subsequently “endorsed” by the 
Department prior to export. Accreditation 
has also been used successfully in other 
areas, such as the inspection of hay for 
export to Japan. 


9.103 The use of accreditation as a 
cost-avoidance tool is being emphasized 
by the Department. It has identified 
several other services it currently provides 
that could be undertaken by “accredited” 
parties. For example, the Department is 
developing accreditation models for 
low-risk nursery plant products, for 
delivering almost all departmental 
activities concerning seeds, for some 
laboratory testing requirements and for 
other activities. A special team has been 
created to standardize the use of these 
models in the Branch by applying 
internationally recognized ISO 9000 series 
of quality standards. All of the 
accreditation initiatives we described are 
expected to be introduced during the 
1996-97 fiscal year. 


9.104 Accreditation is a reflection of 
the changing approach to delivering the 
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Animal and Plant Health Program. In the 
past, enforcement and inspection have 
been delivered by federal inspectors. As 
accreditation becomes more prevalent, the 
focus will change to monitoring and 
compliance. To manage the risks 
associated with this transition 
successfully, the Department will need to 
ensure that its expectations of accredited 
parties are clearly spelled out and that 
strong monitoring and performance 
reporting systems are in place. This is a 
weak area, a fact confirmed by recent 
internal audits of the current accreditation 
programs for both animal health and plant 
protection. In particular, an audit of the 
accreditation of veterinarians cited 
specific weaknesses in monitoring and 
control. As a consequence, a formal 
review of the program has been completed 
and the Department has indicated that a 
modified accreditation program for 
veterinarians is to be implemented in 
April 1996. 


9.105 The Department has recently 
completed a limited-scope “Diagnostic 
Design Project”, which had a mandate to 
develop a new vision, strategic plan and 
program design for diagnostic testing 
activities of the Animal and Plant Health 
Program. The 1994-95 “recoverable” 
costs of these activities were 

$32.2 million, approximately 50 percent 
of which is targeted for recovery from 
industry. This internal review was able to 
identify cash savings of $50,000 annually 
and staff reductions of 10 full-time 
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equivalents, even though cost reductions 
and cost avoidance were not its primary 
thrust. This demonstrates the need to 
conduct an in-depth review of existing 
laboratory operations to determine if 
further savings are possible. 


9.106 The Department should: 


¢ examine all significant aspects of 
its operations to identify additional 
opportunities for cost avoidance and/or 
cost reduction; and 


e establish appropriate systems and 
practices to monitor and control 
activities that are performed by 
“accredited”’ parties. 


Department’s response: The Department 
accepts the recommendation and is 
committed to making further progress 
through its cost avoidance and cost 
reduction initiatives. Initiatives in this 
area will not result in an impediment to 
trade. 


The Department agrees that appropriate 
quality assurance and compliance 
verification mechanisms are necessary for 
all methods of program delivery. The 
adoption of International Standards 
Organization (ISO) and the internationally 
recognized Standards Council of Canada 
(SCC) norms and protocols are being 
pursued. Negotiations are currently under 
way with the Standards Council of Canada 
regarding the accreditation of non-federal 
laboratories. A transparent consultative 
process has been established to provide 
the broadest opportunity for stakeholders 
to challenge operational procedures and 
policies. 
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| ty About the Audit 


Objective 
We conducted this audit with a view to assessing the adequacy of the Department’s practices and identifying 


areas requiring significant improvement in the management of the Animal and Plant Health Program. 


Specifically, we assessed whether: 


e asound basis is used to assess the risk of potential threats to human, animal and plant health and the 
economic consequences; 


e program activities are designed and delivered in a way that deals with the most significant risks and is 
consistent with the mandate and objectives of the Department in this area; and 


e the information to demonstrate the achievement of program objectives and results is readily available and 
reported to Parliament. 


The chapter reports any opportunities we identified to improve planning, conducting or reporting of program 
activities. 


Scope 


| This audit examined the animal and plant health sub-activity of the Inspection and Reowaion Activity 
| referred to in Agriculture and Agri-Food Canada’s Part III Estimates. The Department carries out this 

| sub-activity through two lines of business — Animal Health and Plant Protection. This sub-activity is 

| designed to protect the health of Canada’s animal and plant resources against severe damage caused by 
disease (including those transmittable to humans), pests, weeds and toxic substances. 


In addition, we examined the changes and other major initiatives that have taken place since our last audit of 
this area in 1988. 


Other aspects of the food production process (see Exhibit 9.2) were excluded from the primary focus of this 
audit. For example, we did not examine Food Inspection (an area we ae on in 1994), Veterinary Drugs 
and Pesticides (areas regulated by Health Canada). 


Audit Team: 


Linda Anglin 
Rose Chevrier 
Angus Mickle 
John Rossetti 
Mike Ryan 
Gordon Stock 


For information, please contact Douglas Timmins, the responsible auditor. 
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~ Chapter 10 


Correctional Service Canada 


Rehabilitation Programs for 
Offenders 


The audit work reported in this chapter was conducted in accordance with the 
legislative mandate, policies and practices of the Office of the Auditor General. 
These policies and practices embrace the standards recommended by the Public 
Sector Accounting and Auditing Board ( PSAAB ) of the Canadian Institute of 
Chartered Accountants. 
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Rehabilitation Programs for Offenders 


Assistant Auditor General: Maria Barrados 
Responsible Auditor: David Brittain 


Main Points 


10.1 Part of the mandate of Correctional Service is to rehabilitate offenders in its care. Since most of the 
14,000 offenders currently incarcerated in federal penitentiaries will eventually be released to the community, 
many argue that this role is the most important part of the Service’s mandate. It is particularly important when one 
considers that, historically, 28 percent of released offenders committed new offences after release. 


10.2 Correctional Service spends approximately seven percent of its $1 billion annual budget on programs 
targeted at factors that contribute to criminal behaviour. Traditional programs, such as education, vocational 
training and prison industries, focus on employability. Since 1988, intervention programs have been the main 
focus of the rehabilitation efforts of Correctional Service. These programs focus on factors such as sexual 
deviancy, substance abuse, and antisocial behaviour. Today, Correctional Service has designed an impressive 
series of intervention programs, some of which have received international recognition. 


10.3 We noted weaknesses in the management of the $34 million spent on intervention programs. The Service 
spends a disproportionate amount of its intervention program resources on two contracts without any assurance 
that the right offenders are being treated or any analysis indicating that the results being achieved justify their 
cost. For instance, 17 percent of the expenditure on sex offender programs is spent to treat 20 offenders, while the 
remaining 83 percent is spent on 1,800 offenders. For substance abuse programs, 16 percent of the expenditure is 
spent to treat 100 offenders, while the remaining 84 percent is spent on 5,000 offenders. Another weakness is that 
the Service has not fully established a continuum of programs to properly support offenders in their transition 
from the institution to the community. The Service is currently able to meet only 65 percent of demand for relapse 
prevention programs for sex offenders in the community. 


10.4 There are also weaknesses in the management of the Service’s traditional programs. CORCAN (the 
prison industry operation) is the Service’s most expensive rehabilitation program, using more than $100 million in 
the past four years. CORCAN jobs cost $13,000 per full-time job in 1994-95, while basic education and 
vocational training cost about $7,500 per full-time student. Furthermore, half of the offenders assigned to 
CORCAN did not necessarily need the skills being offered. Another concern is that over 95 percent of the 
Service’s employability resources are focussed in the institutions, leaving very little to help offenders get ajob 
once they return to the community. The Service does not have a coherent strategy to deal with offender 
employability. It cannot make the necessary trade-offs for resource allocation. 


10.5 Correctional Service is committed to reintegration and has been successful in developing a wide range of 
intervention programs. However, there is a lack of attention by senior managers to determining the best match of 
resources to their overall objective of safe reintegration. To achieve this balance, the Service will require better 
information on the cost and benefits of its rehabilitation programs. Without such information, the Service lacks the 
capability to strategically reassess and reallocate its investment in rehabilitation programs. 
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Introduction 


Correctional Service has two jobs — 
incarceration and rehabilitation 


10.6 At any one time, approximately 
14,000 offenders are incarcerated in 
federal penitentiaries. Most will 
eventually be released to the community, 
either on parole or statutory release, prior 
to serving their full sentence or at the end 
of their sentence. Because virtually all 
offenders will ultimately be released, the 
impact of Correctional Service on 
offenders extends beyond incarceration 
(see Exhibit 10.1). 


10.7 Part of the Service’s mandate is 
to rehabilitate the offenders in its care. To 
meet this objective, the Service: 


e assesses offenders when they enter 
an institution to identify the 
“criminogenic” factors that have led to 
their criminal behaviour; 


e assigns offenders to programs based 
on these assessments; 


* assesses whether participating in 
rehabilitation programs has contributed to 
reducing the risk that an offender will 
commit another offence after release; 


e makes recommendations to the 
National Parole Board on the release of an 
offender to the community; and 


¢ provides supervision, further 
assessment and programs in the 
community until the end of the sentence. 


Changing offenders’ behaviour has been 
a goal of corrections for a long time 


10.8 
corrections in Canada, changing the 
behaviour of offenders has been a goal. 
Before Kingston Penitentiary was opened 
in 1835, it was thought that removing 
individuals from the “evil influences” that 
led them to crime and subjecting them to 


From the earliest days of 
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hard work would not only punish them but 
contribute to reforming them as well. 
Over the years, the debate over the role of 
penitentiaries has continued, as has the 
debate over the efficacy of rehabilitation. 


10.9 By the 1980s, the rehabilitation 
programs of the Service consisted mainly 
of education and vocational training as 
well as prison industry jobs. Those efforts 
were supplemented by inmate self-help 
groups such as Alcoholics Anonymous 
and some locally developed programs, 
such as sex offender treatment programs, 
which began at the Regional Psychiatric 
Centres in the Ontario and Pacific Regions 
in the 1970s. At the time, Correctional 
Service saw its role as providing 
rehabilitation opportunities to offenders, 
who could take advantage of them if they 
wished. 


By the 1980s, the 
rehabilitation 
programs of the 
Service consisted 
mainly of education 
and vocational training 
as well as prison 
industry jobs. 


The Service significantly changed its 
approach to rehabilitation in the late 
1980s 


10.10 Starting in 1988, Correctional 
Service fundamentally changed its 
approach to rehabilitation programs by 
implementing what it terms an “active 
interventionist” approach. In 1991, it 


Exhibit 10.1 


The Mandate of Correctional 
Service Canada 


The mandate of Correctional Service is defined by the 
Corrections and Conditional Release Act (1992), which states 
that ’the purpose of the federal correctional system is to 
contribute to the maintenance of a just, peaceful and safe 
society by: 


® carrying out sentences imposed by courts through the safe 
and humane custody and supervision of offenders; and 


@ assisting the rehabilitation of offenders and their 
reintegration into the community as law-abiding citizens 
through the provision of programs in penitentiaries and in 
the community.” 
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Exhibit 10.2 


Factors That Relate to 
Criminal Behaviour 


enunciated four guiding principles on 
which the approach is based: 


e Addressing the problems that have 
led to criminal behaviour is essential to 
reducing recidivism. 


e The environment within the 
institution must be conducive to changing 
an offender’s behaviour. 


e The attitude, values and skills of 
staff are important elements of the 
Service’s effort to change an offender’s 
behaviour. 


e Correctional Service must have an 
organized approach to providing 
rehabilitation programs. 


10.11 The Service has implemented a 
number of programs intended to deal with 
criminogenic factors, that is, those factors 
that research indicates relate to criminal 
behaviour (see Exhibit 10.2). 


10.12 In 1994-95, out of a 
departmental budget of approximately 
$1 billion, Correctional Service spent 
$75 million, or seven percent, on 


programs that deal with criminogenic 
factors. A list of these programs and their 
estimated cost is shown in Exhibit 10.3. 


10.13 In 1994-95, the Service began 
devolving operational responsibility for 
programs from national and regional 
headquarters to local management. As a 
result, national headquarters is responsible 
primarily for developing and 
implementing new programs, carrying out 
program-related research, and allocating 
funds to the regions. Regions are 
responsible for allocating their program 
resources and for training instructors. 
Institutions and parole offices are 
responsible for providing programs and 
for selecting instructors. This devolution 
of authority is still in process. In some 
regions, roles and responsibilities of the 
various players have not yet been fully 
developed. The Service has implemented 
a resourcing model designed to redress 
historical funding imbalances between 
regions. 


10.14 Programs may be delivered in 
either the institution or the community. 


get or hold a job. 


Research indicates that various factors relate to criminal activity. A number of 
these factors are static and cannot be influenced (for example, age of first criminal 
conviction). Others, however, are dynamic, and it is these factors that the Service 
hopes to influence with programs. They include: ; 


@ Employment. Many offenders lack the education or job skills to be able to 


@ Marital/Family. Many offenders lack parenting skills and have relationships 
marked by instability and family violence. 


e Associates/Social Interaction. Many offenders have friends who encourage 
their criminal behaviour or substance abuse habits. 


@ Substance Abuse. Many offenders have significant problems with drugs and 
alcohol. 


@ Community Functioning. Many offenders have difficulties in managing 
their finances and leisure time. 


@ Personal/ Emotional Orientation. Many offenders have serious problems 
with impulsive behaviours, controlling anger and solving problems. In 
addition, a significant proportion of federal offenders have committed sexual 
offences. 


@ Attitude. Many offenders maintain attitudes that are anti-social and/or 
anti-female. 
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Programs in the institution are aimed at 
reducing the risk posed by an offender to a 
level that will make him suitable for 
release. Programs in the community are 
intended to build upon gains that offenders 
have made in institutional programs. 
Community programs also need to deal 
with risks specifically associated with 
being in the community — for example, 
by supporting a sex offender in his efforts 
to avoid relapse. In 1994—95, the 
education and CORCAN (prison industry) 
programs were delivered almost 
exclusively in the institution. As well, 
three quarters of the money spent on other 
programs (such as sex offender and 
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substance abuse programs) was spent in 
institutions, and one quarter was spent in 
the community. 


10.15 In an institution, an offender’s 
day is ruled by the routine of the 
institution (see Exhibit 10.4). On an 
average weekday, an offender has 
approximately six hours during which he 
might take part in activities. Offenders can 
be involved in programs, education, 
institutional employment (working in the 
kitchen, institutional maintenance or 
cleaning), vocational training and 
CORCAN. There are also offenders who, 
for a variety of reasons, are unemployed. 


Exhibit 10.3 


Correctional Service’s Rehabilitation 
Programs and Costs, 1994-95 


Intervention programs that address behaviour: $ millions 
Sex Offender Programs 10 
Substance Abuse 7 
Locally Designed Programs 5 
Living Skills 4 
Other* 8 
Total 34 
*administration, aboriginal programs, family violence programs, 
community psychological counselling 
Traditional programs that address employability: $ millions 
Education LO 
CORCAN (prison industry) 16.6 
Vocational Training Sp 
Total 41.0 
ie Other activities related to rehabilitating offenders: as $ millions 
Mentally Disordered Offenders ies) 
Social and Cultural Programs (visits, hobbycraft, recreation) 125 
Inmate Pay 20.0 
Chaplaincy So 
Total 45.5 
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Exhibit 10.4 


A Typical Inmate Weekday 


Rehabilitating offenders is a difficult 
task, made more difficult by the 
environment of correctional institutions 


10.16 Correctional Service’s 
rehabilitation programs aim to reduce 
criminal behaviour by dealing with the 
factors that have led offenders to commit 
their crimes. Changing behaviour is a 
difficult task in itself, which is made more 
difficult by other factors: 


e By the time individuals are admitted 
into a federal penitentiary, they have often 
developed problems that make them 
highly resistant to change. 


e The sterile and artificial nature of the 
penitentiary environment does not 
necessarily support efforts to rehabilitate. 
Added to this is the pressure of living in 
close confinement with other offenders. 
As well, correctional officers (guards) are 
less supportive of rehabilitation than are 
other groups within the Service (as 
indicated in the Correctional Service staff 
survey). This lack of support can take the 
form of not co-operating with those 
involved in programs and failing to 
reinforce positive behaviours taught in 
Correctional Service programs. 


e According to Correctional Service 
studies, the prevalence of alcohol and 
drugs in penitentiaries may reduce the 
impact of its substance abuse programs. 


7:00 — 

8:00 — 
11:45 — 
13:00 — 
16:30 — 
18:00 — 
22:30 — 
23:00 — 


breakfast 

go to program, work or back to the cell 

return to the cell for count and lunch 

go to program, work or back to the cell 

return to the cell for count and then supper 

go to recreation, cultural events, self-help groups 
night count 


lock-up 


10.17. The Service has attempted, with 
varying degrees of success, to address 
some of these issues. For instance, it 
attempts to encourage program 
participation through inmate pay. The pay 
scale is intended to reward offenders who 
are complying with their correctional 
plans. The service is examining whether 
the current scale sufficiently supports its 
program efforts. It is also attempting to 
reduce the presence of alcohol and drugs 
in institutions through a national drug 
strategy. 


10.18 Complicating decision making in 
institutions are other pressures that often 
conflict with the goals of rehabilitation 
programs. For example: 


e Programs are useful as a way of 
keeping offenders busy in the institution. 
This is particularly true of programs that 
can be provided to a large number of 
offenders over a long period of time (for 
example, education, vocational training 
and CORCAN). As a result, there is 
pressure to accept as many offenders as 
possible in these programs, regardless of 
whether they have criminogenic needs 
that the programs are designed to meet. 


e Although CORCAN’s main purpose 
is essentially rehabilitative, there is 
pressure to be efficient to meet its 
objective of breaking even. This can lead 
to selecting offenders with needed job 
skills to participate in the program rather 
than those who most need the skills being 
taught. This is discussed further in 
paragraph 10.48. 


e Programs are often given to an 
offender only to increase the likelihood of 
parole. For instance, many low-risk 
offenders are seen as suitable for 
rehabilitation programs in the community. 
However, the Service will frequently 
register those offenders for institutional 
programs in order to facilitate their 
application for parole with the National 
Parole Board. This practice leads to an 
overprescription of institutional 
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programs, and ultimately to long waiting 
lists. 


10.19 The Service has put a lot of effort 
into developing its rehabilitation 
programs, some of which have an 
excellent reputation. Research indicates 
that rehabilitation programs can be 
effective; however, it is essential that they 
be run well. As the Service has limited 
resources to devote to programs, it must 
make choices wisely. For example, it 
needs to be very clear about whom to 
treat. Should it take the worst offenders 
(such as those most likely to reoffend 
violently) and focus its efforts on 
minimizing the risk that they will 
reoffend? Or should it take the offenders 
who are most likely to benefit from 
programs, regardless of risk, and focus on 
them? Or should it expose as many 
offenders as possible to as many programs 
as possible and hope for the best? 


Observations and 
Recommendations 


10.20 The audit examined the value of 
the $34 million spent on the Service’s new 
program thrust, which is focussed on 
changing inmate behaviour (addressing six 
of the seven criminogenic factors 
identified in Exhibit 10.2). Here we tried 
to determine if the Service had a clear set 
of guidelines on who was eligible for 
programs; an integrated, cost-effective set 
of programs; information on costs and 
results; and a framework for assessing 
programs and reallocating resources. 


10.21 Second, we looked at the 

$41 million spent on the traditional 
programs of CORCAN, education and 
vocational training, which are focussed on 
employability skills — academic, 
interpersonal and teamwork. These 
programs have been in existence for a 
long time and, essentially, were once the 
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Service’s rehabilitation efforts. Here we 
tried to determine if the Service had a 
clear strategy for dealing with 
employability, and if it had a mechanism 
for allocating its resources among the 
various components. Additional details 
about the audit are presented at the end of 
the chapter. 


Correctional Service 
dramatically changed 
its approach to 
rehabilitation in the 
late 1980s. 


Intervention Programs 


10.22 As noted earlier, Correctional 
Service dramatically changed its approach 
to rehabilitation in the late 1980s. The 
main thrust of its new approach, as 
specified in the Correctional Strategy, was 
the decision to concentrate its limited 
resources on programs that the research 
literature indicated had the most potential 
for reducing recidivism. This involved 
developing programs aimed at those 
offender characteristics related to criminal 
behaviour — such as treatment for sex 
offenders, substance abuse programs, 
family violence initiatives, and living 
skills programs. These programs are 
mainly delivered within the institutions; 
some continue after the offender has been 
released into the community. Some 
intervention programs attempt to influence 
attitudes and behaviour indirectly by 
conveying information; others try to 
change behaviour directly. Some of the 
key programs are described below. 


10.23 Sex offender treatment. A 
variety of sex offender programs are 
offered in 27 institutions. They range from 
intensive (for example, a one-year 
residential program in a psychiatric 
centre) to intermediate (for example, a 
five-month program offered within a 
regular institution by a psychologist) to 
low-intensity programs. The latter are 
typically “relapse prevention” programs, 
which are offered in minimum security 
institutions and prepare offenders for their 
eventual release by both teaching them to 
recognize the factors that led to their 
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offence and helping them to develop the 
skills to manage those factors. 
Community-based relapse prevention 
programs are also available and are 
important in maintaining gains. In 
1994-95 Correctional Service spent 

$10 million to treat approximately 

1,800 sex offenders. See Exhibit 10.5 for 
the evolution of sex offender treatment. 


10.24 Substance abuse. A variety of 
programs are available, but there are two 
key nationally developed programs. One 
is called the Offender Substance Abuse 
Pre-Release Program. It consists of 

26 sessions, each three hours long, and is 
given in an institutional setting. It is 
designed to be followed by the second 
program, called Choices, which is 
delivered in the community. Choices has 
five six-hour sessions followed by weekly 
maintenance sessions for three months. 
Both programs are seen by the Service to 


be appropriate for offenders with 
moderate substance abuse problems. 
There is also a national education program 
for offenders with low-level substance 
abuse problems. Finally, there are some 
locally developed programs in the various 
institutions, based on different 
philosophies of substance abuse treatment. 
In 1994-95 Correctional Service spent 

$7 million on all substance abuse 
programs to treat approximately 

5,000 offenders. 


10.25. Living skills. The living skills 
initiative consists of a series of programs, 
the key component of which is called 
Cognitive Skills. It is a five-year-old 
program designed to change the 
anti-social thinking that led offenders to 
criminal behaviour. This program involves 
36 two-hour sessions. There also is an 
Anger and Emotions Management 
program component, which has recently 
been developed to teach offenders to 


Exhibit 10.5 


Evolution of Sex Offender Treatment in Correctional Service 


Twenty-three years have elapsed since Correctional Service first provided sex offender treatment at the Regional Psychiatric 
Centres in the Ontario and Pacific Regions. The sex offender population has grown tremendously since then, and so has the 
information on characteristics and risk factors specific to the different types of sex offenders. Today there are nearly 5,000 sex 
offenders under Correctional Service jurisdiction, who represent approximately 25 percent of the federal offender population. It 
is the segment that is growing at a faster rate than any other offender group in Correctional Service. Approximately 3,500 of 
these 5,000 offenders are serving their first sentence for a sex offence. In 1991, a census revealed that a majority of the sex 
offenders were convicted of sexual assaults. The average sentence for a sex offender in 1995 was 5 years, compared with 4.2 
years in 1991. The vast majority of sex offenders are male. 


| Sex offenders vary greatly in nature and in degree of dangerousness. The best predictor of sexual reoffence is the individual’s 

sexual offence history. Some are considered much more likely to reoffend (for example, homosexual child molesters) than others 
(for example, incest offenders). Rapists are more likely to use gratuitous violence. Experts agree that sex offenders can never be 
cured, but can be taught to control their impulses. Community follow-up is especially important as they learn to manage their 
deviance in a setting where risk factors are reintroduced, as opposed to the artificial setting of the institution. 


Since the 1970s, the methods used to treat sex offenders have become much more sophisticated. Today Correctional Service uses 
laboratories both to assess the level of deviance and to teach offenders to react in a more acceptable manner to various stimuli. 
Many sex offender programs begin by addressing the denial that is typical of this group. Group therapy is also used, as it allows 
peers and the therapists to confront the offender when he attempts to justify his crimes. Another important feature is training the 
offenders to establish their “crime cycle”, that is, all of the factors that led to their unacceptable sexual behaviour. 


Correctional Service currently has the capability to treat sex offenders at three levels of intervention: intensive, moderate and low 
(which includes maintenance and relapse prevention). Its treatment capacity has increased from 200 in 1988 to 1,800 in 1995. 
This capacity, however, is not keeping pace with the influx of sex offenders into the system. 
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manage harmful emotions and to deal with 
high-risk anger situations. In 1994-95 
Correctional Service spent $4 million to 
treat approximately 6,000 offenders in the 
living skills field. 


10.26 Family violence. In 1995 
Correctional Service estimated that at 
least 33 percent of the offender population 
had indications of family violence in their 
background. To deal with the issue of 
family violence among this group, the 
Service has given priority to developing 
new programs, some within the 
institutions and others in the community. 
In 1994—95 the Service spent 
approximately $1.9 million to treat 
approximately 500 offenders. 


Wide variation exists in the costs and 
methods used to treat offenders who 
pose a similar risk of reoffending 


10.27 The Service’s treatment of sex 
offenders varies considerably from region 
to region. For example, a high-risk sex 
offender may undergo a year-long 
residential program in a psychiatric 
facility in one region, while in another 
region an equally high-risk sex offender 
will receive a six-month, non-residential 
program. A second problem with sex 
offender programs is that the costs vary 
widely for a given level of risk. For 
example, in some moderate-intensity 
programs it costs $2,000 to treat an 
offender, while in other similar programs 
it costs up to $7,000 per offender. Similar 
cost discrepancies can be found in both 
high- and low-intensity treatment. Given 
the wide variation in length, approach 
and, ultimately, cost, the Service needs to 
concentrate its effort on those sex offender 
treatment programs that are most 
cost-effective. 


10.28 Another anomaly in sex offender 
programs is the disproportionate amount 
of resources spent on a few offenders, 
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without any assurance that they are the 
right offenders and that the program is 
achieving its intended results. 

Seventeen percent ($1.7 million) of the 
Service’s total sex offender program 
budget is used in one region, on a contract 
to treat up to 20 sex offenders per year. 
The remaining 83 percent ($8.3 million) 
of the sex offender program budget is used 


to treat 1,800 sex offenders per year. Seventeen percent of 


the Service’s total sex 
offender program 

budget is used to treat 
up to 20 sex offenders. 


10.29 The same type of problem exists 
with substance abuse programs, where a 
disproportionate amount of resources is 
spent in one region on a contract to treat a 
small group of 100 offenders who need 
substance abuse treatment. The cost of 
treating these offenders ($1.1 million) 
represents 16 percent of the total 
expenditures for substance abuse. The 
remaining 5,000 offenders receive only 
$5.9 million for treatment. 


10.30 Correctional Service has greatly 
expanded its knowledge and expertise 
since these contracts were initiated in the 
1980s. Since that time, the Service has 
also developed a significant array of 
programs designed to address similar 
types of criminogenic needs. However, to 
date it has not determined whether the 
contracts are cost-effective in relation to 
alternative programs within the Service. 


The Service has not consistently ensured 
continuity between institutional and 
community programs 


10.31 To reduce the risk to the 
community, appropriate treatment or 
assistance for offenders must continue to 
be available in the community after they 
have been released. This is critical to 
reducing recidivism because, upon return 
to the community, offenders are once 
again confronted with those factors that 
initially led to their criminal conviction. 
The goal is for offenders to use skills 
taught within the institution to deal with 
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Exhibit 10.6 


Family Violence Programs 
Expenditure by Region 1994-95 


“hazardous” situations in the community. 
One example of such treatment is called 
“relapse prevention”, which is a 
recognized approach to treating sex 
offenders and substance abusers. 


10.32 Despite the fact that relapse 
prevention for sex offenders in the 
community is a highly effective 
risk-management tool, there is an 
estimated shortfall in this “maintenance” 
treatment available in the community. 
Currently, 35 percent (or approximately 
400 released offenders) are not receiving 
relapse prevention treatment. 
Furthermore, this shortfall is most 
pronounced in the two regions with the 
largest proportion of sex offenders. 


10.33 We also noted that a continuum 
of programs for family violence is 
lacking. In 1990, approximately 

$9 million was allocated to Correctional 
Service for use over four years to develop 
and implement family violence programs. 
Since that time, this initiative has gone 
from an experimental to an official 
program. However, ensuring that 
offenders are properly identified and 
receive intervention programs for family 
violence has been a difficult task. Of a 


% to % to 
| Region ($ 000) Institution | Community 
| Pacific 267 Ie) 87 
Prairies 345 39 61 
Ontario 639 pe 78 
Quebec 368 81 19 
Atlantic S| 0 100 
National $1,896 32% | 68% 


Source: Correctional Service Canada data 


sample of 80 offenders now in the 
community who were identified in 1995 
as needing family violence programs, we 
found that only a few had participated in 
suitable programs in the institution and 
none in the community. In fact, nothing in 
the files on most of these offenders 
indicated that their parole officers knew 
that family violence was an issue. 


10.34 A comparison of expenditures on 
family violence programs shows 
considerable differences in the amount of 
money that different regions spend on 
providing a continuum of family violence 
treatment from institution to community. 
For example, while most regions spend 
the majority of their funds in the 
community, the Quebec region invests 
only 19 percent in the community (see 
Exhibit 10.6). 


10.35 Another problem with continuity 
of programs is in the design of programs 
delivered in the community. Programs 
originally developed for institutions are 
difficult to deliver effectively in the 
community because their structure reflects 
the artificial environment in institutions. 
However, we found that most programs 
delivered in the community were 
essentially duplicates of those offered in 
the institutions, rather than designed as a 
“booster/follow-up” program to be 
delivered in the community. We did note, 
however, that the Service has recently 
developed a Cognitive Skills booster, 
although this program is still not available 
nationally. 


Recruitment and support of program 
instructors require further emphasis 


10.36 The Service has established 
standard training programs and selection 
criteria for instructors in nationally 
designed programs. The Service believes 
that selecting the right people (those who 
best meet the selection criteria) will 
ensure that it will have instructors with the 
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skills to influence offenders. Correctional 
Service then trains the instructors in the 
subject matter they will be required to 
teach. However, each region has adopted a 
different salary classification range for 
instructors, resulting in a wide 
discrepancy in salaries. Correctional 
Service has not fully considered the 
implications of this inconsistency on its 
rehabilitation programs. 


10.37 As well, we noted that instructors 
receive limited support for their classroom 
work. Assessment is done by national or 
regional instructors who review 
videotapes of delivered sessions. Although 
Correctional Service is decentralizing this 
function, more ongoing support is 
necessary, given the difficulty of changing 
the behaviour of offenders in the 
correctional setting. 


The management of intervention 
programs requires greater attention 


10.38 The Service’s ability to intervene 
and change criminal behaviour has grown 
tremendously in the last 10 years. It 
developed a conceptual model based on 
research literature and many elements of 
this model are now in place. However, we 
concluded that there is no management 
framework for intervention programs that 
would enable senior management to 
reassess and strategically reallocate funds 
in order to reduce risk. Without this 
management framework, the Service does 
not know if it is making the best use of its 
funds, in keeping with its overall objective 
of safe reintegration. This lack of a 
framework is illustrated by our audit 
findings: 


¢ different treatment approaches for 
offenders with similar risk profiles; 


e expensive programs conducted for 
small groups of offenders without 
knowledge of their cost effectiveness and 
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results when compared with other 
Correctional Service programs; and 


¢ an incomplete array of programs to 
manage the transition of the offender from 
institution to community. 


10.39 Correctional Service should 
ensure, in keeping with the principles in 
its Correctional Strategy, that its 
resource allocation is based on cost and 
sound information on the benefits of its 
programs, both among similar types of 
programs and across all rehabilitation 
programs. 


Correctional Service’s response: We agree 
with this recommendation. Over the past 
seven years, we achieved an enormous 
growth in the variety and volume of 
programs. We did so by launching new 
programs in institutions that volunteered 
to be the front runners or where external 
sources of expertise were most available. 
As a result, some institutions advanced 
more than others. 


We have begun the necessary 
rationalization of resources. At the macro 
level, a resourcing formula has been in 
place for a couple of years. We have also 
begun a program-by-program review to 
ensure that delivery can be made 
regardless of location. The first programs 
subject to this review have been cognitive 
living skills, moderate-intensity substance 
abuse, and relapse prevention for sex 
offenders. 


The Service’s ability to 
intervene and change 

criminal behaviour has 
grown tremendously in 


Employability Programs ee eae 
10.40 Research indicates that offenders 
who find good jobs upon release are less 
likely to be reincarcerated. In a recent 
study, the Conference Board of Canada 
identified employability skills as key to 
success in the Canadian work force: 
academic skills (communication, thinking, 
learning), personal management skills 
(attitudes, behaviours, responsibility, 
adaptability) and teamwork skills. To help 
offenders acquire these skills, the Service 
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has education, vocational programs and 
CORCAN. The latter is a special 
operating agency that provides jobs to 
offenders in prison, in a setting that is 
intended to model a real work situation. In 
addition, the Service has limited 
employment counselling and job 
placement services in the community. The 
cost associated with these programs varies 
significantly. As we have noted earlier, 
these programs face contradictory 
pressures. The emphasis on rehabilitation 
can be at odds with the need to keep 
offenders busy and, in the case of 
CORCAN, the pressures of trying to run 
an efficient and financially viable 
operation. 


10.41 We expected that management 
would have reviewed these various 
programs and determined whether they 
formed a coherent national strategy 
designed to deal with offender 
employability. We considered this review 
particularly important given that 
governments are eliminating their 
employment programs for offenders in the 
community. 


CORCAN 


10.42 Offender employment within the 
federal prison system dates back to the 
19th century. In the early years, the 
purpose of prison employment was to 
reduce idleness among offenders. By 
1960, Correctional Service began 
focussing on the potential of prison 
employment as a rehabilitation activity. 


10.43 In 1981 Correctional Service 
created a national prison industry. The 
trademark CORCAN was used to identify 
goods and services produced using inmate 
labour. CORCAN was made a special 
operating agency in 1992. It operates 
farms and workshops in 33 of the 


10-16 


Service’s 43 prisons and uses inmate 
labour to produce goods and services for 
sale to Correctional Service and other 
federal departments and public agencies. 
Products and services include agricultural 
commodities, construction, garments, 
office furniture and data entry. When it 
became a special operating agency, 
CORCAN was given the authority to enter 
into joint ventures and marketing 
arrangements with the private sector. 


10.44 ‘In addition to revenue from the 
sale of goods and services, CORCAN 
receives an annual training and 
corrections fee from Correctional Service 
(see Exhibit 10.7). This fee, $16.6 million 
in 1994-95, is intended to offset the cost 
associated with training offenders and 
compensate CORCAN for operating 
within a prison setting. When CORCAN 
was made a special operating agency, it 
also obtained access to a $45 million 
revolving fund to provide working capital 
and investment funds to assist in an 
expansion of its rehabilitation mandate. 


10.45 According to CORCAN’s 1992 
special operating agency charter, the 
agency is to: 


e offer offenders work-related training 
and work experience in accordance with 
needs identified in their correctional 
plans; and 


e provide a wide range of 
employment-related services in the 
community to help released offenders to 
re-enter the labour market and reintegrate 
into society. 


In addition to the training and corrections 
goals, the new mandate set out these 
objectives: 


e increase offender participation in 
CORCAN; and 


e be financially self-sustainable. 
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CORCAN is not meeting its training 
and corrections goals 


10.46 CORCAN has been unable to 
meet its expanded goals since becoming a 
special operating agency with a revolving 
fund in 1992. It has not performed as 
expected in areas related to its 
rehabilitation and corrections mandate: 
the number of training jobs it provides to 
offenders; targeting offenders who lack 
employability skills; and providing 
employment-related services to help 
released offenders reintegrate into the 
community. 


10.47 For example, employment of 
offenders at CORCAN has remained 
relatively constant at approximately 
1,700 full-time jobs. This number had 
been forecast to increase to 2,000 by 
1995-96. 


10.48 CORCAN’s mandate is to 
provide offenders with training and 
employment in accordance with 
employability needs identified in their 
correctional plan. These needs are 
identified during the offender intake 
process, at which time a correctional plan 
is prepared. At five institutions we 
examined a sample of offenders assigned 
by the Service to work at CORCAN. We 
found that, in the majority of cases, 


Training fee from CSC 
$ 16.6 million (33%) 
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neither the files nor, more specifically, the 
correctional plan indicated that the 
offenders needed the training in 
employability skills provided by 
CORCAN. Furthermore, we found that 
nearly half the offenders in our sample 
had had a good employment history 
before being imprisoned, or that they had 
either held consecutive CORCAN jobs 
over a number of years or were working 
with CORCAN in areas where they 
already had job experience. 


CORCAN has not 
performed as expected 
in areas related to its 
rehabilitation and 
corrections mandate. 


10.49 CORCAN has a mandate to 
provide a wide range of employment- 
related services to assist offenders being 
released or living in the community. These 
services are to range from job search and 
counselling to community-based 
employment projects and workshops. 
Currently, CORCAN offers these services 
to offenders in only 12 locations across 
the country. It spent $1.6 million on these 
projects in 1994-95, half of it in one 
region. 


CORCAN is not meeting its goal of 
self-sustainability 


10.50 In the three years ending 

31 March 1995, CORCAN had net 
operating losses of $7.3 million. Further 
losses of up to $2 million are forecast by 
CORCAN for 1995-96. In particular, 


Exhibit 10.7 


CORCAN - Sources of 


Sales to CSC Revenues 1994-95 


$ 16.4 million (33%) 


Other Sales 
$17.0 million (34%) 


TOTAL: $50 million 
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CORCAN’s Industries (manufacturing) 
business line has sustained significant 
operating losses each year. As a result, 
CORCAN has been drawing down its 
revolving fund more quickly than was 
originally expected. The $45 million 
revolving fund was expected to have a 
balance of only $7—10 million at 

31 March 1996. 


10.51 
financial performance are varied and 


The reasons for this poor 


require management’s attention: 


e In order to maintain offender 
employment levels, CORCAN operates 
14 manufacturing facilities despite low 
demand for some of their products. 
Furthermore, work in a prison setting 
entails a high supervisor/offender ratio 
and, in some cases, low offender 
productivity. 


e CORCAN has experienced declining 
demand in the traditional federal 
government market for office furniture. To 
date, the introduction of new products and 
services has been insufficient to address 
these operating losses. Thus, CORCAN 
remains reliant on the Service for the 
majority of its operating revenue. 


e A 1974 Cabinet decision, which was 
reaffirmed by the Treasury Board 
ministers in 1995, encouraged federal 
departments to purchase from CORCAN 
when appropriate. CORCAN stated that 
the response has been lukewarm. 


¢ There are costs that are not directly 
attributable to units of sales. These costs 
are high relative to sales — $10 million 
on $30 million sales in 1994-95 — and 
overall operating margins are insufficient 
to cover them. 
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Vocational Training 


The Service needs to assess the potential 
of vocational training to meet 
employability needs 


10.52 Correctional Service spends 
approximately $5.2 million per year on 
vocational programs. According to its 
latest figures, 3,572 offenders were 
registered in vocational programs in 
1994-95, which is the equivalent of 
728 full-time student positions, at a unit 
cost of approximately $7,200 each. 
Seventy-two percent of these students 
successfully completed their course of 
study. 


10.53 = Traditionally, Correctional 
Service has offered a wide range of 
vocational training programs in its 
institutions. These include auto 
mechanics, auto body repair, small engine 
repair, sheet metal, upholstery, welding 
and woodworking. Some institutions have 
eliminated many or all of their vocational 
programs because of difficulty with 
certification, the high cost of some 
programs, the length of time needed for 
completion (when compared with the 
average length of stay in an institution for 
a typical offender), and the difficulty in 
anticipating which skills the job market 
will be seeking. 


10.54 Although some institutions have 
cut vocational programs, others have 
taken the initiative to develop and 
implement simpler training programs. For 
example, following an analysis of its 
vocational programs, one institution 
developed an inexpensive course using 
maintenance of the institution to teach 
inmates custodial/janitorial skills such as 
equipment maintenance, painting and 
drywall installation. The course is 
inexpensive, has a short time frame, and 
provides inmates with skills that are 
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appropriate to the typical inmate’s 
aptitudes and for entry level into the work 
force. We have been told that inmates who 
have completed the course have had some 
success in finding related jobs. However, 
ventures such as these have generally 
remained local. While they appear to be 
worth further investigation for broader 
use, there is very limited evaluation 
carried out within Correctional Service, 
either at the local or regional/national 
levels, to determine which vocational 
programs are the most cost-effective. In 
some cases, vocational training may have 
possibilities as an alternative to 
CORCAN. Its advantage seems to be its 
lower cost and flexibility to keep pace 
with changing local circumstances, both 
inside the institution and in the 
community. 


Education 


Education is delivered consistently well 
across the Service 


10.55 = In 1994-95, Correctional Service 
reported that 70 percent of offenders 
entering institutions tested below Grade 8 
and 86 percent below Grade 10. Their 
average level was Grade 7.4. The Service 
views low education as a factor related to 
criminal behaviour. Currently, its policy is 
to provide all inmates with Grade 10 
equivalency. Generally, basic education 
needs are addressed before proceeding 
with vocational training or CORCAN 
jobs. The Service spends about 

$12.5 million per year on basic education 
(Grades 8 and 10 programs combined). 
The approximately 7,000 enrollments per 
year in Correctional Service schools are 
equivalent to 1,700 full-time students, at 
an average cost of about $7,400 per 
student. 


10.56 Offenders can complete up to 
Grade 12. However, it is not a 
requirement, and the program relies 
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heavily on self-study by the inmate. 
Currently, the equivalent of about 

771 full-time students are pursuing Grade 
12 studies, at a cost of about $7,200 per 
full-time equivalent or $5.6 million per 
year. Inmates generally pay for their own 
post-secondary education, unless it can be 
demonstrated that the education addresses 
a specific criminogenic need. 


10.57 Correctional Service requires that 
the education programs delivered in 
institutions meet provincial accreditation 
standards. In some cases, the institution or 
its contractor is accredited as a school, 

and the principal has the authority to grant 
grade level accreditation to the students. 
In other cases, the students write 
provincially set exams to qualify for the 
grade level equivalency. A few institutions 
are in transition from one system to 
another, but inmates in Correctional 
Service institutions are generally receiving 
education that is recognized and valid 
upon their release. 


Strategy for Addressing Offender 
Employability 


Resources used to address employability 
are not well managed 


10.58 There are significant variations in 
costs among CORCAN, vocational 
training and education. For example, both 
education and vocational training are 
much less expensive per inmate than 
CORCAN. Furthermore, experience to 
date indicates that education and 
vocational training are given to those who 
need it, whereas half of the offenders in 
CORCAN shops do not necessarily need 
the training CORCAN offers. 


10.59 CORCAN jobs cost more than 
other comparable employability training. 
The cost to Correctional Service for a 
CORCAN job — based on the training fee 
of $16.6 million for 1,688 jobs — was 
$9,800 in 1994—95. This does not include 
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The Service is not able 
to make the trade-offs 
necessary to address 
employability as a 
criminogenic factor. 


CORCAN’s operating loss, which was 
financed by drawing down the revolving 
fund. Taking these operating losses into 
consideration, the cost of a CORCAN job 
was approximately $13,000 in 1994-95. 
(CORCAN forecasts a cost per job of 
approximately $11,000 for 1995-96.) By 
comparison, vocational training and basic 
education cost about $7,500 in 1994-95. 


10.60 The Service does not have an 
adequate continuum to assist offenders in 
their transition from the institution to the 
community. For instance, we found that 
there is only limited assistance to help 
offenders locate and keep a job, once 
released into the community. As well, 
employment counselling or job placement 
services offered by the Service or by other 
agencies is sporadic. Under fiscal 
restraint, Human Resources Development 
Canada is having difficulty maintaining 
the traditional level and form of assistance 
for ex-offenders. In addition, the federal 
government’s decision to withdraw from 
purchasing labour market training may 
leave a significant void in the range of 
services to offenders. 


10.61 The Service is not able to make 
the trade-offs necessary to address 
employability as a criminogenic factor. 
Correctional Service has the option of 
spending its resources on any combination 
of education, vocational training, 
CORCAN or job search assistance. It 
needs a framework for making these 
choices and a process to ensure that they 
are periodically reassessed. For example, 
it could change its education policy and 
raise the education level of offenders to 
Grade 12. It could put more effort on job 
search and counselling, or more emphasis 
on vocational training. It could rationalize 
those CORCAN operations (both business 
lines and sites) that are the most 
expensive. 


10.62 The Correctional Service 
should evaluate the costs and benefits 
related to its expenditures in CORCAN, 
education and vocational training to 
establish clear guidelines on who should 
be trained and establish which 
programs are most cost-effective. 


Correctional Service’s response: We 
agree. Greater efforts should be made to 
assess the respective contributions to 
offender employability of prison 
industries, education and vocational 
training. However, they are not 
necessarily true alternatives since many 
offenders need adult basic education as a 
prerequisite for entry into vocational 
training or prison industry. It should also 
be noted that most labour market 
observers argue that adaptability rather 
than specific job skills is the most 
important determinant of employability in 
a rapidly changing economy. Education 
and actual job experience are more likely 
to bring this about. 


Management of Rehabilitation 
Programs 


10.63 The rehabilitation initiatives of 
Correctional Service over the past decade 
represent a significant accomplishment. 
The Service has put a great deal of effort 
and resources into improving programs 
since 1988. Its efforts to reduce recidivism 
by good rehabilitation programs are 
laudable. 


10.64 Correctional Service has recently 
shifted responsibility for many aspects of 
rehabilitation programs to its regions, 
institutions and parole districts. Each 
region has established committees (or 
councils) to plan and develop as well as 
oversee program delivery. While there is 
some variation among regions, the 
committees are responsible for functions 
such as identifying new program needs 
and training program instructors. In some 
regions, the mandates for these 
committees have not yet been formalized, 
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and the committees are not formally 
connected to senior regional management. 
Within institutions there are different 
organizational structures. The other 
rehabilitation programs are distinct from 
CORCAN, with a further separation 
between the intervention programs and the 
educational and vocational programs. 


10.65 The budget for rehabilitation 
programs is allocated to the regions under 
a process that was introduced in 1994 to 
address the long-standing problem of 
uneven allocation of resources among 
regions. The regions have different 
approaches to funding and delivering 
programs and do not have control over the 
entire rehabilitation program budget; the 
CORCAN budget, for example, is 
separately controlled. As a result, the 
management of rehabilitation programs is 
diffused across a multitude of centres. 
There is little capacity to manage the 
diversity of programs from an overall 
perspective. This contributes to offenders 
with identical needs at different 
institutions or in the community receiving 
different levels of programming. 


Better measurement of program costs 
and effectiveness is needed 


10.66 With the Service’s decentralized 
approach to rehabilitation programs, 
management at all levels needs clear 
goals, targets and measurement systems 
that provide results-based information on 
the extent to which criminogenic needs 
are being addressed and results achieved. 
Correctional Service has limited cost 
information available on its rehabilitation 
efforts. Recently, the Service has 
developed a series of independent 
databases for some activities, such as sex 
offender programs, institutional programs, 
community programs and education 
programs. These databases contain 
information on costs and participation 
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rates. However, they are difficult to use in 
an integrated way because they were 
developed independently for different 
purposes. An initiative is under way to 
develop a national performance 
measurement system for rehabilitation 
programs. Progress to date has been slow, 
but plans call for the completion of the 
framework in 1996. 


10.67 In our 1994 Report chapter on the 
National Parole Board, we indicated that 
“Board members...often do not know what 
(if any) benefit ...the offender has received 
from programs.” This information is 
important both for making release 
decisions and for prescribing or 
continuing post-release treatment in the 
community. According to our discussions 
with Correctional Service and National 
Parole Board management and our review 
of recent Correctional Service 
investigations, closing the information gap 
remains a challenge. Without this 
information, Correctional Service’s ability 
to manage risk is hampered. 


10.68 A further concern is the lack of 
information about the overall 
effectiveness of many of the rehabilitation 
programs in contributing to the safe 
reintegration of offenders into the 
community. In order to make the best 
possible release decisions affecting the 
risk to the public, it is critical to know 
“what works” and “what does not work” 
in rehabilitation programs. 


10.69 To date, the effectiveness of only 
a few of the Service’s rehabilitation 
programs has been measured. However, 
the Service has made progress in this area. 
We examined three recently completed 
evaluation studies. In two studies, we 
concluded that, while there was general 
merit, the scope and methodology were 
not sufficient to produce a reliable and 
comprehensive picture of the programs’ 
effectiveness. In the third study, the 
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Correctional Service 
lacks the management 
framework to 
effectively guide the 
delivery of 
rehabilitation 
programs. 


Service used a much more rigorous 
methodology. Although we found some 
problems with the application of the 
methodology, we commend the Service 
for its increased efforts in this area. Given 
the importance of studies like these in 
contributing to decisions affecting release 
to the community, further efforts are 
required that will increase the scope and 
rigor of these evaluations. More 
important, the Service needs to broaden its 
efforts to include assessing the 
effectiveness of combinations of programs 
provided to offenders, as well as the 
measurement of the relative impacts of 
different programs targeted at the same 
criminogenic need. 


The Service needs to address the overall 
management of rehabilitation programs 


10.70 As noted earlier, we expected the 
Service to have a management framework 
that would allow it to: 


e setaclear policy direction on how 
rehabilitation goals are to be achieved; 


e establish clear guidelines on who 
should be treated, and the level of 
intervention or treatment required; 


e establish an integrated, cost-effective 
set of programs; 


¢ monitor the operational performance 
of the program as well as the results being 
achieved (at the overall and individual 
offender levels); and 


e reassess its rehabilitation programs 
and reallocate resources on the basis of 
changing needs, priorities and results. 


10.71 
Service: 


In our audit we noted that the 


¢ spends a disproportionate amount of 
its intervention programs budget on two 
contracts, without any assurance that the 
right offenders are being treated or any 
analysis that indicates that the results 


being achieved by these services justify 
their cost; 


e has not established a continuum of 
programs from the institution to the 
community to properly support offenders 
in their transition to the community; 


e spends more money on its traditional 
employability programs than it does on its 
newer research-based intervention 
programs; 


e spends a large amount of its 
employability resources on CORCAN, 
which was offered approximately half of 
the time to offenders with minimal 
employability needs; 


e lacks an adequate information base 
to assess and compare the performance of 
its various programs; and 


e lacks the knowledge of how effective 
most of its programs are. 


10.72 Our observations led us to 
conclude that Correctional Service lacks 
the management framework to effectively 
guide the delivery of rehabilitation 
programs in accordance with its overall 
objective of reduced recidivism. We 
believe that it is now time to strengthen 
the management of its rehabilitation 
programs. 


10.73 Correctional Service Canada 
should develop the capability to 
strategically reassess its rehabilitation 
program expenditures and reallocate its 
funds as necessary. The Service should 
develop performance measures so that 
management at all levels can truly 
manage the rehabilitation effort. 


Correctional Service’s response: The 
Correctional Service agrees 
wholeheartedly with this recommendation. 
It is obviously the goal to be pursued. 
However, despite our close ties with most 
of the major prison services in the world, 
we are not aware of any that have found 
fully satisfactory ways of evaluating the 
relative values of programs or that have 
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come up with more powerful performance _ financial coding structure, enhancements 
measures than unit costs and participation __ to the management monitoring system, 
rates. This is not an argument for better program referral criteria, and the 
complacency; we are determined to establishment of an ongoing national 
remain in the forefront of this field and, to council on correctional programs. The 
this end, have launched a number of steps project team charged with this work is due 
such as a manual on program standards to report in September 1996. 

and guidelines, refinements to the 
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ty About the Audit 


Scope 


This audit focussed on the activities and services of Correctional Service Canada that address the 
criminogenic needs of its offender population. Specifically, we looked at the activities related to $75 million 
of expenditures on education, vocational training, CORCAN, and substance-abuse, sex offender and living 
skills programs. Excluded from the scope of our examination were the Service’s recent initiatives in the area 
of rehabilitation programs specifically designed for women and aboriginal offenders. 


Objective 


The objective of the audit was to assess how well the rehabilitation process is being managed. Specifically we 
assessed whether: 


e there is clear policy direction on how rehabilitation goals are to be achieved; 
e there are clear guidelines on who should be treated, and the level of intervention or treatment required; 
e there is an integrated, cost-effective set of programs; 


e there is a process to monitor the operational performance of the program as well as the results being 
achieved (at the overall and individual offender levels); 


e the Service periodically reassesses its rehabilitation program efforts and reallocates resources on the basis 
| of changing needs, priorities and results. 


Audit Team 


Tom Beaver 

Estelle Charette 

Bill Johnson 

Chantal Michaud 
Marion Smith 

Mary Louise Sutherland 
Jocelyne Therrien 
Frank Vandenhoven 


For information, please contact David Brittain, the responsible auditor. 
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